Which Stage of the Foundation Model Lifecycle Is Yours

The situation

A homeware retailer sells around 40,000 products online. Three people in the merchandising team write the product descriptions and get through about sixty a week between them. Stock arrives faster than that, so roughly 9,000 live products show a supplier spec line under the photo and nothing else.

The proposal is an assistant that takes the product attributes and the supplier spec and returns a description in the retailer’s own voice. Three people would build it three different ways. The first would call a model on Amazon Bedrock behind a carefully written prompt and have something running in a fortnight. The second would train a model on the 12,000 descriptions the team has written over four years. The house voice is particular, and generic copy reads like every other homeware site. The third has read about training a model on the whole catalogue plus a decade of customer reviews, and asks why the company would build on somebody else’s model at all.

Nobody disagrees about what the feature should do. They disagree about how far back into the model’s own history the team should reach.

What actually matters

Foundation models [FMs] are large models trained once on broad data and then adapted to many tasks, rather than built one per task. The sequence that produces one, puts it into service and keeps it useful has a name in AWS’s vocabulary: the FM lifecycle. Its stages run in roughly this order:

• Data selection. Choosing and preparing the corpus a model learns from, and deciding what to leave out.
• Model selection. Choosing which model, or which architecture and size, to build on.
• Pre-training. The long training run that turns a broad corpus into general-purpose weights.
• Fine-tuning. Continuing training on a smaller, task-specific or domain-specific set so the model behaves the way one organisation wants.
• Evaluation. Measuring whether the output is good enough, against something more specific than a good feeling in a demo.
• Deployment. Putting the model behind an interface an application can call, with the capacity, latency and cost that implies.
• Feedback. Watching what real users do with the output, and feeding that back into the next round.

Every one of those stages happens for every FM application, including the one that ships in a fortnight. Data selection and pre-training for a model on Bedrock were done by the model provider, at a scale no retailer would fund, and the result sits in the weights. Skipping a stage does not remove it from the lifecycle; it changes who ran it and who is able to change it.

The argument in the room is about where the team enters, and therefore which stages it staffs, funds and gets telephoned about when the output goes strange. Four of the seven never move. Model selection, evaluation, deployment and feedback belong to the retailer in every proposal on the table, because the brand on the page is the retailer’s whichever model wrote the words. The three that do move are data selection, pre-training and fine-tuning, and taking on any of them shifts the boundary left.

Each of those three is a claim about data. Fine-tuning claims the organisation already holds enough examples of the behaviour it wants, consistent enough to learn from. Twelve thousand descriptions written by three people over four years, either side of a rebrand and a category expansion, are not twelve thousand examples of one voice. A training run on them reproduces the inconsistency faithfully. Pre-training claims something far larger, which is why it stays with model providers. And the stage that produces training data as a by-product is feedback: every edit a merchandiser makes to a generated description is a labelled example nobody had to commission.

What we’ll filter on

1. Ownership: which lifecycle stages does the team run itself, and which stay with the model provider?
2. Data on hand: does the organisation already hold examples of the wanted behaviour, in enough volume and consistent enough to train on?
3. Labelling effort: if it does not, how many people and how many weeks would producing them take?
4. Time to first usable output: a fortnight, a quarter, or longer?
5. Cost shape: a per-token bill that tracks usage, a one-off training run, or capacity billed by the hour whether or not anyone calls it.
6. Accountability when quality drifts: who investigates, and what are they actually able to change?

The landscape

Four entry points, ordered by how much of the lifecycle the team takes on. The first three all build on a model somebody else pre-trained. The fourth starts from nothing.

A pre-trained model as it comes, on Amazon Bedrock

The retailer picks a model from the Bedrock catalogue, writes a prompt, passes the product attributes and the supplier spec in the request, and reads the description out of the response. Data selection and pre-training were the provider’s work. Fine-tuning does not run at all. What the team owns is model selection, evaluation, deployment and feedback, and the whole thing is an API call from day one.

Behaviour is steered through the prompt and through what gets put in front of the model at call time: instructions, a handful of approved descriptions as examples, the product’s own attributes. That covers more ground than teams expect. The cost shape is per token, so a quiet month costs less than a busy one. Switching models is a change of identifier plus a fresh round of evaluation.

Customising the model on Bedrock

Bedrock will also produce a customised copy of a supported base model from data the retailer supplies. Three methods are on offer. Supervised fine-tuning uses labelled examples, pairs of input and the wanted output. Reinforcement fine-tuning replaces those pairs with a reward function, written as an AWS Lambda function, that scores each response. Distillation generates training data from a larger teacher model and fine-tunes a smaller student on it. Bedrock once also offered continued pre-training on unlabelled domain text; it is no longer in the customisation documentation, so treat a plan that depends on it as out of date. In every case the team has taken on data selection and a training run, and the boundary has moved two stages left.

The mechanics are compared in the customisation options; at this level what matters is the ownership. A customised model is one only the retailer has. Only the retailer can evaluate it, only the retailer knows what went into it, and every future base-model upgrade means another training run.

Models from SageMaker JumpStart on your own endpoint

Amazon SageMaker JumpStart offers pretrained models that can be deployed, or fine-tuned and then deployed, onto an Amazon SageMaker AI endpoint inside the retailer’s own account. The lifecycle stages look much like the Bedrock customisation route. One difference shows up in the operations rota rather than the diagram: the endpoint is infrastructure the team sizes, scales, patches and pays for by the hour. Where the weights should live, and what that changes, is worked through in the comparison of the two homes for the same model.

This is the entry point for a model that is not in the Bedrock catalogue. It also fits a licence or residency rule that requires the weights in one account, or a latency profile that needs dedicated capacity. It is a heavier commitment than a managed endpoint, and worth taking on only when one of those conditions is real.

Pre-training a foundation model from scratch

Data selection, pre-training and everything after, all of it in-house. At retailer scale this is not selectable. AWS puts the cost of developing a foundation model from scratch at millions of US dollars, and gives BLOOM as the worked case: 384 Nvidia A100 GPUs running for three and a half months. Add a curated corpus at a scale no retailer holds, and a team who have done it before. The retailer’s entire archive of house copy is a rounding error against that corpus. Worth naming so that it can be ruled out with a number rather than a shrug, and so that the stages the provider ran on the team’s behalf are visible.

Evaluation

Side by side

Entry point	Data selection	Pre-training	Fine-tuning	Needs your own training data	No servers to operate	Output in a fortnight	Model swap without retraining
Pre-trained FM on Bedrock	✗	✗	✗	✗	✓	✓	✓
Customised model on Bedrock	✓	✗	✓	✓	✓	✗	✗
JumpStart weights, own endpoint	✓	✗	✓	✓	✗	✗	✗
Pre-trained from scratch	✓	✓	✓	✓	✗	✗	✗

Four stages have no column at all. Model selection, evaluation, deployment and feedback would carry a tick in every row, and a stage that is yours in all four options cannot separate them. What the table separates is the three movable stages. Read down them and the first row is a mirror of the last. Each stage the retailer takes on is one it also has to fund, staff and repeat on every model upgrade.

The second and third rows differ in one place, and that place is not a lifecycle stage. Both fine-tune, both need the data, and both turn a change of base model into another training run. One leaves a managed endpoint to call and the other leaves a server to run.

Where a team should enter

The order of the gates carries the argument. Evaluation sits at the top because until a prompted model has been measured against real products, nobody in the room knows whether there is a gap to close. The data gate sits second because a training run on inconsistent examples produces an inconsistent model and a fortnight of nobody understanding why. Only after both of those does the question of where the weights live become worth an afternoon.

The solution

Enter at model selection. Take a pre-trained model on Bedrock, own evaluation, deployment and feedback, and leave data selection, pre-training and fine-tuning where they are until something forces the boundary left.

Model selection is a first move, not a final one. Shortlist two or three models, hold the prompt constant, and compare them on the retailer’s own products rather than on a leaderboard. Evaluation settles the argument that started the meeting. Assemble a held-out set of a hundred products whose descriptions the brand team has already approved, generate against each shortlisted model, then have brand mark every result accept or reject with a reason. Running that as a scored job rather than a spreadsheet makes it repeatable. It gets re-run on every model change for as long as the feature exists.

Deployment for this workload is a Bedrock batch inference job that runs overnight against the backlog and a per-product call for new stock, with a merchandiser approving before anything goes live. Batch jobs read their prompts from Amazon S3 and write responses back to S3, and they do not run against a provisioned model, which is one more reason to stay on the prompted route. Keep the model identifier and the prompt in configuration rather than in code, so changing models stays a one-line change in practice. Then build the feedback stage properly: capture the merchandiser’s edit alongside the generated text and the reason for the rejection. That does two jobs at once. It shows where the model is weak this month. It also accumulates the paired examples a fine-tune would need, so the option to move left stays open without anyone commissioning a labelling project. Wiring that loop from users back to the model is the work that makes the next decision an easy one.

Hold fine-tuning until evaluation shows a gap that prompting and added context cannot close. Be specific about what the gap is. Two kinds of failure look identical in a demo and call for opposite fixes. Wrong facts, a dimension the model stated that appears nowhere in the spec, are a context problem: the fact was never in front of the model, and training will not put it there reliably. Wrong voice, wrong structure, wrong length, consistently, across every model tried, is the failure that fine-tuning addresses, because it is about form rather than knowledge. Teams reach for a training run to fix invented product facts more often than any other mistake in this space, and it does not work.

Two consequences of customising are worth knowing before signing up to it. The first is that a customised model needs its own inference setup, and the general route is Provisioned Throughput, billed hourly for as long as it exists on a no-commitment, one-month or six-month term. The bill changes from per-token to capacity by the hour whether the batch runs or not, and a feature with lumpy overnight usage can cost more customised than prompted. Custom model deployment is the alternative and keeps the per-token shape, but it is narrow: US East (N. Virginia) and US West (Oregon) only, and only for a short list of base models, currently the Amazon Nova family and Meta Llama 3.3 70B Instruct. Check that list before assuming the token bill survives customisation. The second consequence is that a customised model is pinned to the base model version it was trained from. When the provider ships a better one, the prompted route picks it up after an evaluation run. The customised route needs another training run.

Worked example

The retailer takes the first gate seriously and spends a fortnight on it. Two models from the Bedrock catalogue, one prompt containing the house style rules and six approved descriptions as examples, and the product’s attributes and supplier spec passed in each call. A hundred products across four categories, generated by both, shuffled, and handed to the brand lead with no indication of which model produced what.

The better model comes back at 78 accepted out of 100. The 22 rejections sort into two piles rather than one. Fourteen are factual: a drawer depth that appears in no source, a fabric composition that contradicts the spec, a care instruction with no origin at all. Eight are voice: correct, useful, and reading like a catalogue rather than like the retailer.

Fourteen against eight decides the next move. The factual failures trace back to the supplier spec arriving as a PDF that the prompt was summarising badly. The fix sits in what goes in front of the model rather than in the weights: parse the spec into fields and pass the fields. That takes four days and clears eleven of the fourteen. The eight voice failures cluster in one category, and adding two examples from that category to the prompt clears six of them.

Which leaves five failures out of a hundred, no training run, no dataset, no data selection, and a feature live against the backlog in the sixth week. The merchandisers’ edits are captured from the first day it runs. By the time anyone revisits fine-tuning there will be a few thousand paired examples of the current house voice, rather than four years of two.

What’s worth remembering

1. The FM lifecycle runs for every FM application, and its stages are data selection, model selection, pre-training, fine-tuning, evaluation, deployment and feedback; picking a service picks which of them the provider already ran.
2. Model selection, evaluation, deployment and feedback are yours in every option, so the real choice is only about data selection, pre-training and fine-tuning.
3. Fine-tuning is a claim that you already hold consistent examples of the behaviour you want; without them a training run reproduces the inconsistency.
4. Measure with evaluation before customising, and separate wrong facts from wrong voice: the first is fixed by what you put in front of the model, the second is what fine-tuning is for.
5. Pre-training from scratch runs into millions of US dollars and months on a GPU cluster, on AWS’s own figures, so at ordinary application scale it stays with the model provider.
6. Build the feedback stage on day one, because the edits users make are the labelled dataset any later fine-tune would otherwise have to commission.