Exam-style
A retailer runs an internal assistant on a foundation model so support staff can ask about company policy. The refund policy changed last month: the returns window went from 30 days to 14. The assistant still answers 30 days. Its prompt is standing instructions plus the staff member's question, and nothing else. What is the cheapest change that makes it answer correctly, and keeps it correct at the next change?
Reveal the answer
B. Retrieve the current refund policy text into the prompt on each call, with an instruction to answer from it and to say when it does not cover the question
The prompt does not contain the new policy. It carries standing instructions and a question, so the only refund rules available are whatever the pre-training data held, which predates the change. Nothing contradicts 30 days, so the model returns the most plausible continuation. Deciding what goes into the context window on each call is context engineering. Retrieving the relevant policy sections into each prompt means an amended policy updates every answer on publication, because each call reads the policy as it stands. Fine-tuning trains on labelled examples and adjusts the model’s parameters. It needs data preparation, a training job, an evaluation and a deployment, so it takes days, runs up a training bill, and is stale at the next policy change. Facts that change belong in the context; behaviour and format that stay constant are the candidates for fine-tuning. Temperature shapes the probability distribution over the next token, so a low setting returns the same wrong number more consistently. A larger context window raises the token ceiling per call, and an empty window holds no policy text until the application puts it there. Few-shot examples show the shape of a good answer, not the facts in it, and examples written around the old 30-day rule repeat the wrong number.
Q. An internal assistant answers refund questions with a returns window that was retired last month. Its prompt is standing instructions plus the question. Cheapest fix that also survives the next policy change?
A. Put the current policy text in the prompt. Retrieve the sections relevant to the question, include them in the context window, and instruct the model to answer from that text. Context engineering is deciding what goes into the context on each call.
Why? Nothing in the prompt states the change. Standing instructions plus a question leave the refund rules to the pre-training data, whose most plausible continuation is 30 days. Retrieval puts the current text on the call, so an amended policy updates every answer the day it is published. Fine-tuning needs a data set, a training job and a deployment, and goes stale at the next change. Facts that move belong in the context; behaviour and format that hold still are what fine-tuning is for. Temperature only varies the output, so a lower setting returns the same wrong number more consistently. A bigger context window is empty until the application fills it. More few-shot examples are prompt engineering, shaping the answer rather than supplying its facts, and any example written before last month repeats 30 days.