Lab: The Capstone

This is the last of the ten labs that build everything by hand against the model API. The scaffolding is gone. You get a requirement and a test, and you write the whole handler. The full lab is in lab-10-capstone.zip.

Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper, a standing backstop that auto-deletes any lab you forget to tear down after 24 hours.

The requirement

Build a Greenbox support assistant that takes a question and returns {"answer", "sources", "guarded"}, and is:

• grounded: answers only from the five-document corpus, retrieving the relevant documents and citing them as sources;
• honest: says it does not know when the corpus does not cover the question;
• guarded: names the guardrail on every generation call and blocks financial advice, reporting guarded: true when stopReason comes back guardrail_intervened.

The acceptance test

scripts/test.sh scores three checks out of three:

What’s provided

A Lambda whose role holds bedrock:InvokeModel and bedrock:ApplyGuardrail, and the corpus. The guardrail is an AWS::Bedrock::Guardrail (financial advice denied, email and phone redacted) with an AWS::Bedrock::GuardrailVersion publishing version 1, since the guardrail resource itself only ever reports DRAFT. Its id and version arrive as environment variables. src/handler.py is a skeleton with the requirement in its docstring. You write the body; solution/handler.py is there when you want to compare.

How the pieces fit

Nothing here is new. Retrieval is Lab 05 (_embed, _cosine, retrieve). The guardrail is Lab 02 (guardrailConfig on the Converse call, reading stopReason). Grounding is a system prompt telling the model to answer only from the context and to say so when the context does not cover the question. The capstone is assembling them:

hits = retrieve(question, k=2)
context_block = "\n\n".join(f"[{d['id']}] {d['text']}" for d in hits)

response = _bedrock.converse(
    modelId=MODEL_ID,
    system=[{"text": "Answer only from the provided context. If it is not there, "
                     "say you do not know. Do not give financial advice."}],
    messages=[{"role": "user", "content": [
        {"text": f"Context:\n{context_block}\n\nQuestion: {question}"}]}],
    inferenceConfig={"maxTokens": 400, "temperature": 0.2},
    guardrailConfig={"guardrailIdentifier": GUARDRAIL_ID,
                     "guardrailVersion": GUARDRAIL_VERSION, "trace": "enabled"},
)
guarded = response.get("stopReason") == "guardrail_intervened"

Deploy and prove it

cd lab-10-capstone
# write src/handler.py first
./scripts/deploy.sh
./scripts/test.sh        # aim for Acceptance: 3/3
./scripts/teardown.sh

What the track added up to

Across ten labs you built each piece by hand: a model call and its IAM, a guardrail as a separate versioned control, structured output through tool schemas, conversation memory in a session store, retrieval as embed-compare-rank, a tool the model calls and your code executes, a data-quality gate before ingestion, text-to-SQL with a read-only guard, an evaluation harness with an LLM judgeLLM-as-a-judgeUsing a second model, prompted with a rubric, to score another model’s output when there’s no exact answer to diff against., and finally all of it at once.

A model is one component. The retrieval, the safety, the permissions, the data quality, the evaluation, and the operations around it are what turn a demo into something you can put in front of customers.

What’s worth remembering

1. A production GenAI feature is a system, not a model call; the model is one part, and grounding, safety, permissions, data quality, and evaluation are the rest.
2. Grounding has two halves, and you need both every time: retrieve the right context, and instruct the model to answer only from it and to say so when the context runs out.
3. Safety is a separate, versioned control applied on every call, not a line in the prompt you hope holds.
4. Cite the sources and report when the guardrail acted, so every answer is auditable.
5. An acceptance test turns “it seems to work” into a number you can defend, so you can change the system and see straight away whether it still works.
6. Build each piece so you understand it, then reach for the managed service that runs it for you at scale.