Exam-style
A retailer runs an internal assistant on a foundation model so that support staff can ask questions about company policy in plain English. The refund policy changed last month: the returns window went from 30 days to 14, and opened boxes are now excluded. The assistant still answers 30 days, confidently, with no sign of doubt. Its prompt today is a short set of standing instructions plus the staff member's question, and nothing else. What is the cheapest change that makes it answer correctly, and keeps it correct at the next policy change?
Reveal the answer
B. Retrieve the current refund policy text and place it in the prompt on each call, with an instruction to answer from the supplied policy and to say so when the policy does not cover the question
The assistant has never been shown the new policy. Its prompt carries standing instructions and a question, so the only refund rules available to it are whatever it absorbed during pre-training, which is a snapshot of the public internet from before the change and knows nothing about this retailer anyway. Nothing in the prompt contradicts the 30-day answer, so the model produces the most plausible continuation it can and sounds certain doing it. Deciding what goes into the context window on each call is context engineering, and the change here is to put the current policy text in front of the model: look up the relevant policy sections for the question being asked, include them in the prompt, and instruct the model to answer from that text and to say when the text does not cover the question. Publishing an amended policy then updates every answer, because each call reads the policy as it stands. Fine-tuning adjusts the model’s weights on a training set of examples. It needs data preparation, a training job, an evaluation and a deployment, which is days of work and real money, and it is stale again the moment the policy moves, so the whole cycle repeats. Facts that change go in the context; behaviour and format that stay constant are the candidates for fine-tuning, which is why fine-tuning is a reasonable answer to a complaint about house tone or ticket format, and a poor one to a complaint that the assistant does not know this month’s rules. Temperature controls how much randomness there is in token selection. At zero the assistant gives the same wrong answer every time, in a steadier voice. A larger context window raises the ceiling on how many tokens one call may carry. An empty larger window still contains no policy, and a window is filled by the application, not by the model reaching out for documents. Few-shot examples are prompt engineering, teaching the shape of a good answer rather than supplying the facts in it, and examples written around the old 30-day rule actively teach the wrong number.
Q. An internal assistant answers refund questions with a returns window that was retired last month, and sounds sure of itself. Its prompt is standing instructions plus the question. Cheapest fix that also survives the next policy change?
A. Put the current policy text in the prompt. Retrieve the sections relevant to the question, include them in the context window, and instruct the model to answer from that text. Context engineering is deciding what goes into the context on each call.
Why? The model was never told. Standing instructions plus a question leave the refund rules to whatever pre-training left behind. The most plausible continuation of a refund question is a 30-day answer, delivered with confidence. Retrieval puts the current text on the call, so an amended policy updates every answer the day it is published. Fine-tuning costs a data set, a training job and a deployment, and goes stale at the next change. Facts that move belong in the context; behaviour and format that hold still are what fine-tuning is for. Temperature only changes how varied the wording is, so a lower setting returns the same wrong number more consistently. A bigger context window is empty room until the application fills it. More few-shot examples are prompt engineering, shaping the answer rather than supplying its facts, and any example written before last month teaches 30 days.