Exam-style
A grocery retailer runs a customer assistant on a foundation model in Amazon Bedrock. It answers questions about delivery windows, substitution rules and what happens when an item is out of stock. Operations rewrite those documents every Monday: cut-off times move, suburbs are added and dropped, and the substitution policy changes with the season. The assistant is quoting windows that were withdrawn weeks ago. The team has costed a monthly fine-tuning run on the current policy documents and wants sign-off. Which approach fits, and why?
Reveal the answer
B. Retrieval Augmented Generation (RAG) over an Amazon Bedrock Knowledge Base built on the policy documents, reindexed whenever operations publish a change
Sort the options by what they change. Fine-tuning and continued pre-training change the model: they adjust weights so that behaviour, tone, format or domain vocabulary shift. Retrieval changes what the model is given at answer time, which is what decides whether it can state this week’s cut-off. Facts that move weekly belong in an index that can be rewritten on Monday morning, not in weights that take a training job, an evaluation and a deployment to move. A monthly cycle is also wrong by construction here: for three weeks out of four the model would be authoritative about withdrawn windows, and each run costs a data-preparation effort and a training bill for an artefact that is stale on arrival. Amazon Bedrock Knowledge Bases is the managed form of the retrieval route: point it at the documents in S3, it chunks and embeds them into a vector store, and at answer time it fetches the passages that match the customer’s question and places them in the prompt alongside it. Supplying facts in the prompt this way is in-context learning, which teaches nothing permanent and needs nothing permanent, because the next call reads the index as it stands. Republishing a policy and reindexing updates every answer from that moment, and the retrieved passage doubles as a citation the customer support team can check. Raising the context window and pasting everything in is the same idea done expensively: token cost on every call for documents the customer did not ask about, a hard ceiling once the policy set grows, and the model still has to find the relevant clause in a wall of text. Temperature governs how varied the token choices are, so a lower setting returns the withdrawn cut-off more consistently rather than less often. Continued pre-training suits a model that does not speak the domain, absorbing vocabulary and phrasing from a large corpus; it is a poor way to install a specific time that will be different next week.
Q. A grocery assistant quotes delivery windows and substitution rules that operations rewrite every Monday, and it is weeks behind. The team has costed a monthly fine-tuning run. Which approach fits?
A. Retrieval Augmented Generation (RAG) over an Amazon Bedrock Knowledge Base, reindexed whenever operations publish a change.
Why? Fine-tuning changes behaviour and format; retrieval changes what the model knows at answer time. A weekly fact belongs in an index you can rewrite on Monday, not in weights that need a training job, an evaluation and a deployment to move, and that are three weeks out of date for most of the month. A Knowledge Base chunks and embeds the documents into a vector store, then puts the matching passages in the prompt with the question, which is in-context learning: nothing permanent is taught and nothing permanent needs to be. A bigger context window with everything pasted in pays tokens on every call for policy nobody asked about. Temperature only steadies the wording, so the withdrawn cut-off comes back more consistently. Continued pre-training teaches a domain’s vocabulary, not this week’s timetable.