Exam-style
A team chose a foundation model from the Amazon Bedrock catalogue, wrote prompts around it, wired it into a customer-facing help assistant and shipped. The assistant has been live for a month. Under every answer there is a thumbs-up and a thumbs-down button, and the team stores each rating alongside the prompt and the answer that produced it. Roughly one answer in twenty comes back with a thumbs-down, and most of those are questions about refunds, where the assistant quotes a policy that changed last year. Which stage of the FM lifecycle is the team running, and what does the collected signal feed?
Reveal the answer
C. Feedback, and it feeds evaluation first, then fine-tuning when the ratings show a gap that is consistent rather than scattered
The stages of the FM lifecycle run data selection, model selection, pre-training, fine-tuning, evaluation, deployment, feedback. Collecting what real users think of real answers is feedback, the last stage and the one that closes the circle. What it feeds depends on what the ratings say. First stop is always evaluation: the ratings turn into a scored set of real prompts, and every future prompt change or model change gets measured against it instead of against a demo. Fine-tuning is the second stop, and only when the gap is systematic. Here it is not: the refund answers are wrong because the policy changed and the model is quoting an old one, which is a retrieval and context problem that a corrected help-centre article fixes in an afternoon. Fine-tuning a model to memorise a policy that will change again next year buys a training run and a fresh problem. The three wrong answers all misfile the activity. Data selection is choosing the corpus a model learns from, which happens before training rather than after launch. Deployment is putting the model behind an interface an application can call and keeping it there, so it covers the assistant running but not the ratings coming back. Pre-training is the long training run that turns a broad corpus into general-purpose weights; almost no team ever runs it, because a foundation model arrives already pre-trained and thousands of thumbs-downs are nothing next to a corpus measured in trillions of tokens. Model selection is a real stage and the ratings do inform it, but swapping the model is a response to the measurement rather than the name of the activity producing it.
Q. A live assistant is collecting thumbs-up and thumbs-down from users, stored with the prompt and the answer. Which stage of the FM lifecycle is that, and what does it feed?
A. Feedback. It feeds evaluation first, and fine-tuning after that, when the ratings show a consistent gap rather than scattered complaints.
Why? Learn the seven in order: data selection, model selection, pre-training, fine-tuning, evaluation, deployment, feedback. Feedback is the one that runs forever, because it starts the day the thing goes live and never finishes. Ratings on real answers, stored next to the prompts that produced them, become a scored set you can re-run whenever anything changes, which is why evaluation is the first place they go. Fine-tuning is the next place, and it earns the training run only when the ratings cluster: one topic, one tone, one shape of answer, wrong over and over. Scattered thumbs-downs are usually a prompt or a stale document, so fix those before you reach for customisation. The tempting wrong answer is pre-training, because ratings feel like training data. Almost nobody runs pre-training. A foundation model arrives with that stage already paid for by whoever built it, and the stages a team actually owns start at model selection.