Make It Smart
Add an LLM Feature with Vertex AI
Your site stops being a brochure and starts doing something.
You build
An LLM feature in the site.
Core concept
Calling an LLM, prompting, Vertex AI.
Your portfolio is live. Now you'll give it a brain. In this phase you enable Vertex AI — Google Cloud's managed home for foundation models like Gemini — and add a backend endpoint that calls an LLM, then wire an interactive AI feature into the site itself.
A natural default: an “Ask my resume” chat box. But the pattern is generic — swap in a project-idea generator, a tone-adjustable bio, or a “summarize my experience for a recruiter in X industry” button. Whatever you pick, the shape is identical — and it's the shape of nearly every LLM feature you'll ever build.
Why it matters in industry
Almost every product now has an “AI feature.” Knowing how to call a model from a backend safely — keeping keys server-side, controlling cost, guarding against bad output — is table-stakes engineering, not research.
Example — A customer-support tool adds a “summarize this ticket” button. It's a single server-side model call behind an endpoint — structurally identical to the feature you'll wire into your site here.
Concept
How an LLM feature actually works
Three moving parts: a frontend (a box where the user types something and sees a response); a backend endpoint (a route like /api/ask that receives the text, builds a prompt, and calls the model — this must live on the server); and Vertex AI (you send messages, you get a generated response).
The one security rule: keep model calls server-side. Never call the model directly from browser JavaScript, and never ship credentials to the client. The browser talks to your backend; your backend talks to Vertex AI.
Browser
user types a question
Your backend
/api/ask · server-side only
Vertex AI
Gemini generates
The browser never talks to the model directly. Credentials stay server-side; only your backend calls Vertex AI, and the answer returns back along the same path.
Step 1 · Enable Vertex AI
Turn on the API
Add Vertex AI to the services you enabled in Phase 0.
“Enable the Vertex AI API (aiplatform.googleapis.com) on PROJECT_ID and tell me in plain terms what Vertex AI is.”
Gemini models are served through Vertex AI. For a chat feature, a fast, inexpensive Gemini Flash model is the right default.
gcloud services enable aiplatform.googleapis.comStep 2 · Permissions
Let your Cloud Run service call the model
Here's where Cloud Run shines. Your service runs as a service account, and you can grant that identity permission to use Vertex AI directly — so your code authenticates automatically, with no key anywhere. Add this to the Terraform in your /terraform folder, as a normal pull request.
Because the code runs as that service account, Google's client libraries pick up credentials automatically (“Application Default Credentials”). You never write a key into your app.
“In my /terraform folder, add a dedicated service account for the Cloud Run site, grant it the Vertex AI User role, and attach it to the Cloud Run service. Put it on a branch and open a PR so it deploys when I merge. Explain why running as a service account means no API key in my code.”
resource "google_service_account" "site" {
account_id = "portfolio-run"
}
resource "google_project_iam_member" "vertex" {
project = "PROJECT_ID"
role = "roles/aiplatform.user"
member = "serviceAccount:${google_service_account.site.email}"
}
# attach that identity to the Cloud Run service
# template { service_account = google_service_account.site.email ... }Step 3 · The Backend Call
Add the model-calling endpoint
Install the Google Gen AI SDK and add a server route. The core call is small; the point is understanding request → prompt → response.
“Add a server-side API route to my site (e.g. /api/ask) that takes a question, calls Gemini on Vertex AI with the Google Gen AI SDK, and returns the answer. Keep it strictly server-side. Walk me through how the model call works.”
npm install @google/genaiimport { GoogleGenAI } from '@google/genai';
const ai = new GoogleGenAI({
vertexai: true,
project: process.env.GOOGLE_CLOUD_PROJECT,
location: 'YOUR_REGION',
});
export async function POST(req: Request) {
const { question } = await req.json();
const resp = await ai.models.generateContent({
model: 'gemini-2.5-flash',
contents: `Answer as ME, based on my resume below.\n` +
`RESUME:\n${MY_RESUME}\n\nQUESTION: ${question}`,
});
return Response.json({ answer: resp.text });
}Step 4 · The Prompt
Make it answer well
The quality of the feature is mostly the quality of the prompt. Give it a role and context (“Use only the resume below. If something isn't covered, say so rather than inventing it.”). Guard against hallucination explicitly — an LLM will confidently fabricate a job you never had if you let it. Set the tone to match your site's voice.
A preview of Phase 3
Notice you're pasting your whole resume into every prompt. That works for one short document. But answering from dozens of documents by stuffing them all into every prompt breaks down fast. That problem is exactly what Phase 3 (RAG) solves.
Step 5 · Wire It In & Ship
Connect the frontend and deploy
Add a chat box, deploy through the same PR-merge loop, then test against edge cases: an off-topic question, an empty input, something not in your resume. Confirm it degrades gracefully.
“Add a chat box to my site that POSTs to /api/ask and streams the response back token by token. Then put it all on a branch and open a PR so it deploys when I merge.”
Set GOOGLE_CLOUD_PROJECT as an env var on the Cloud Run service (in your Terraform template block).
Cost & safety guardrails
A public LLM endpoint can be abused. Before sharing widely: add basic rate limiting, cap the response length, and keep an eye on your budget alert. A Gemini Flash model answering short questions is cheap, but an open endpoint with no limits is how a free trial disappears overnight.
Done when
- Vertex AI enabled; a Gemini model chosen.
- Cloud Run service account granted aiplatform.user — no keys in code.
- A server-side endpoint that calls the model.
- A prompt that answers as you, without hallucinating.
- An interactive AI feature, live on your site, with basic guardrails.
1.Where must the call to the LLM live, and why?
2.On Cloud Run, how does your code authenticate to Vertex AI with no API key?
Answer every question correctly to complete this phase.