Lab 04 — Give a Bedrock chatbot a memory

Scaffold: 4/5. The table is built and wired; you write the load-and-save logic that turns single calls into a conversation.

The scenario

A support chatbot answers each message perfectly and forgets it instantly. Ask it your name, then ask it back, and it has no idea, because a model call has no memory: every request starts from a blank slate. To hold a conversation you replay the earlier turns on each call, and that transcript has to live somewhere between requests. This lab puts it in DynamoDB, keyed by session.

The requirement

Two messages in the same session behave like a conversation:

You:  My name is Sam and my favourite colour is teal.
Bot:  Nice to meet you, Sam.
You:  What is my name and favourite colour?
Bot:  You are Sam and your favourite colour is teal.

What’s provided

• template.yaml — the Lambda, a DynamoDB table keyed by session_id (with TTL so old chats expire), and an IAM policy granting Bedrock invoke plus GetItem/PutItem on that table. The table name arrives as an environment variable.
• src/handler.py — parses session_id and prompt, and makes a single-turn call. The gap is the memory.
• solution/handler.py — the reference answer, including a turn cap so the history cannot grow without limit.
• scripts/ — deploy, test (two turns in one session), teardown.

Your task

In src/handler.py:

1. Load the history for session_id from DynamoDB before the call (the item stores the message list as JSON in messages).
2. Append the new user turn, call the model with the full list, then append the assistant reply.
3. Save the updated list back.

Keep every entry in the Converse messages shape: {"role": "user"|"assistant", "content": [{"text": "..."}]}.

Run it

./scripts/deploy.sh
./scripts/test.sh          # states a fact, then asks for it back
./scripts/teardown.sh

What success looks like

Before you wire it, the second answer has no idea what your name is. After, it does, and the turns count climbs across calls in the same session. Start a new session (SESSION=other ./scripts/test.sh) and the memory is separate, because it is keyed by session id.

If it fails

• Still forgetting — check you are reading the item before the call and writing it after, and that you send the whole messages list, not just the latest turn. Read with ConsistentRead=True: the default read can miss a turn written a second earlier, so the memory looks broken when the code is right.
• ValidationException about roles — the transcript must start on a user turn and alternate user and assistant after that. Append the assistant reply after each call, and if you cap the history, trim the window so it still opens on a user turn.
• History grows huge / context errors — cap the replayed turns (the solution keeps the last twenty, then drops a leading assistant turn so the replay still starts with the user). Unbounded history eventually overflows the context window and inflates cost.

Reveal the solution

SRC=solution ./scripts/deploy.sh && ./scripts/test.sh

What you just learned

• A model is stateless; conversation memory is something you build by replaying the transcript, not a feature of the call.
• Short-term memory is the recent turns you replay; a store like DynamoDB keyed by session is the usual home, with a TTL so it does not accumulate.
• The transcript has to stay in the model’s message shape and alternate roles, or the call is rejected.
• Memory has a cost: every replayed turn is input tokens, so cap or summarise the history rather than letting it grow forever. That trade, replay recent turns versus summarise older ones, is the seed of long-term memory.

Next

Lab 05 — Stand up a vector store and answer from your own documents. The scaffolding drops again: you wire retrieval into the chat, grounding answers in a corpus instead of the model’s own memory.