Exam-style
A grocery retailer runs a customer assistant on a foundation model in Amazon Bedrock. It answers questions about delivery windows, substitution rules and out-of-stock handling. Operations rewrite those documents every Monday: cut-off times move, suburbs are added and dropped, substitution rules change with the season. The assistant is quoting windows withdrawn weeks ago. The team has costed a monthly fine-tuning run and wants sign-off. Which approach fits, and why?
Reveal the answer
B. Retrieval Augmented Generation (RAG) over an Amazon Bedrock Knowledge Base built on the policy documents, reindexed whenever operations publish a change
Fine-tuning and continued pre-training change the model, adjusting its parameters so that behaviour, tone, format or domain vocabulary shift. Retrieval changes what the model is given at answer time, and the prompt is where this week’s cut-off has to come from. Weekly facts belong in an index that can be rebuilt on Monday, not in weights that need a training job, an evaluation and a deployment to change. A monthly run leaves three weeks in four quoting withdrawn windows, and AWS bills training by tokens processed plus monthly storage per custom model. Amazon Bedrock Knowledge Bases is the managed form: it chunks the S3 documents, embeds them into a vector index, retrieves the chunks nearest the question at query time, augments the prompt and cites the source. Pasting the whole set into a larger context window answers correctly too, but bills tokens on every call for clauses nobody asked about. Temperature shapes the probability distribution over the next token: a lower value returns the withdrawn cut-off more consistently, not less often. Continued pre-training adapts a model to a domain’s language, not to a time that changes next week.
Q. A grocery assistant quotes delivery windows and substitution rules that operations rewrite every Monday, and it is weeks behind. The team has costed a monthly fine-tuning run. Which approach fits?
A. Retrieval Augmented Generation (RAG) over an Amazon Bedrock Knowledge Base, reindexed whenever operations publish a change.
Why? Fine-tuning changes behaviour and format; retrieval changes what the model is given at answer time. A weekly fact belongs in an index you can rewrite on Monday, not in weights that need a training job, an evaluation and a deployment to move, and that are three weeks out of date for most of the month. A Knowledge Base chunks the documents, embeds them into a vector index, then puts the matching passages in the prompt with the question, which is in-context learning: nothing permanent is taught and nothing permanent needs to be. A bigger context window with everything pasted in bills tokens on every call for policy nobody asked about. Temperature only steadies the wording, so the withdrawn cut-off comes back more consistently. Continued pre-training teaches a domain’s vocabulary, not this week’s timetable.