Which Kind of Training a Foundation Model Needs

The situation

A specialist marine insurer writes hull and cargo policies and settles roughly 22,000 claims a year. Eighteen months ago it put an assistant on Amazon Bedrock in front of its claims desk. A surveyor’s report arrives as a PDF, and the assistant drafts a first-pass assessment note that an adjuster then edits and signs.

The cheap improvements have all been made. Policy wordings and the current clause library sit behind a knowledge base, so the assistant answers from documents rather than from memory, and a glossary of about 400 trade terms rides along in the prompt. Two rounds of prompt work have been done. The assistant is right far more often than it was, and the adjusters still rewrite almost every draft, because it produces four flowing paragraphs when the desk wants eight labelled fields with the applicable exclusion stated first. The team has worked up the customisation ladder and reached the rung where the model itself changes.

Four proposals are now in front of the head of claims. Train a marine model of the company’s own. Fine-tune on the 60,000 historical assessment notes in the claims system. Feed the model forty years of surveyor reports so it learns to talk like a surveyor, an archive the data team has since measured at about 145 million tokens once the repeated headers and standard clauses are stripped out. Or take the answers the assistant already produces and train a smaller model to produce them for less. Somebody has also asked whether the adjusters could just rank drafts and have the model learn from that. Each of those is a different job needing a different kind of data, and nobody has said which data the company can actually supply.

What actually matters

Start with what a foundation model already is when it arrives, because that decides what is left to do. Pre-training is the run that builds general capability from an enormous unlabelled corpus, trillions of tokens of ordinary text, on a cluster of accelerators, over months. It is done by the model provider before the model reaches a catalogue. Nobody in this building will run one. Naming it matters anyway, because everything else on the list is defined by starting from its output rather than from randomly initialised weights.

That reuse has a name. Transfer learning is the idea that knowledge learned on one task can be carried into another as a starting point, instead of learning the second task from nothing. Fine-tuning is an instance of it, and AWS lists transfer learning alongside instruction tuning, adapting models for specific domains, and continuous pre-training as methods for fine-tuning a foundation model. Read that list as one family rather than four rivals: they all continue training a model that has already been pre-trained, and they differ in what they feed it.

Fine-tuning continues training on a comparatively small labelled set, where each item is a prompt paired with the completion you wanted for it. Because the examples are labelled, the model learns the shape of a good answer: the structure, the ordering, the tone, the length, the task. Instruction tuning is the form this usually takes, fine-tuning on instruction-and-response pairs so that a model which merely continues text becomes one that follows an instruction. Hundreds or a few thousand pairs move a model a long way, which is why the expensive part of fine-tuning is people writing and checking examples rather than compute.

Continuous pre-training feeds raw unlabelled text from one domain into a model that has already been pre-trained. AWS also writes it as continued pre-training, and the two names mean the same job. Nothing is labelled, so nothing in the data shows the model what a good answer looks like. What shifts is the vocabulary it handles fluently: the drug names, the vessel types, the clause language of one industry. No labelling effort goes into it, and it takes raw text at a scale most companies overestimate until they count.

Distillation is the odd one out because it changes neither knowledge nor behaviour. A large, capable teacher model is run over your prompts, and its outputs become the training data for a smaller student model. What comes back behaves much like the teacher on that narrow task at lower cost and lower latency. It changes the model’s size, so it applies once the answers are already good.

Set those four against each other and the discriminator is the data. Labelled pairs teach behaviour, raw domain text teaches vocabulary, a teacher’s outputs move settled behaviour onto a smaller model, and human rankings teach preference. Adapting models for specific domains is the goal all of this serves, and it names no single technique. Whether it lands on instruction tuning or on continuous pre-training depends on whether the gap is what the model says or how it says it.

What we’ll filter on

1. What the job changes: general capability, behaviour and format, domain vocabulary, or model size.
2. What kind of data it consumes: raw unlabelled text, labelled prompt-and-completion pairs, a teacher model’s outputs, or human preference rankings.
3. How much of that data it needs, and whether the company actually holds that much.
4. How much human labelling effort has to happen before any training job starts.
5. Who can realistically run it: a model provider, or this team as a customisation job in its own account.
6. What has to be redone when the base model is upgraded.

The landscape

Four training jobs, in the order the model itself would meet them.

Pre-training

Randomly initialised weights, a curated corpus of trillions of tokens scraped and filtered from broad text, and a training run measured in accelerator-months. The output is a base model that can continue text plausibly on almost any subject and follow no instructions at all. This is where general capability comes from, and it is out of reach for anyone who is not a model provider. Worth pricing once so it can be ruled out with a number. Worth naming too, so that “train our own model” gets unpacked in the room rather than nodded through, since it usually turns out to mean fine-tuning.

Fine-tuning, and instruction tuning as its usual form

Take a pre-trained model, hand it labelled examples of the task you care about, and continue training. Each example is a prompt and the completion that should have come back. On Amazon Bedrock this is a model customisation job over a dataset in Amazon S3, and AWS names the method supervised fine-tuning; on Amazon SageMaker AI it is a training job over a model from Amazon SageMaker JumpStart, whose instruction-based option takes the same prompt-and-response examples. Either way the output is a private custom model that answers the way the examples answered.

Instruction tuning is fine-tuning where the pairs are instructions and their responses, which is how a base model comes to follow an instruction rather than carry on the text. The models in a catalogue have usually had this done to them already. A team fine-tuning today is generally teaching a house-specific version of a task the model can already do adequately: this form, these fields, this order, this length. Data volumes are small by training standards and large by human standards. A few hundred to a few thousand pairs is typical, and every one of them needs someone with the domain judgement to say what the right answer was.

Continuous pre-training

The same continuation of training, without labels. You supply raw domain writing, and the model’s handling of that language improves. It addresses the case where a model reproduces “general average” or “inherent vice” less reliably than it reproduces ordinary English. It fixes nothing about output format, because nothing in the data says what a good answer looks like.

Check where the job can run before proposing it. Amazon Bedrock’s documented customisation methods are supervised fine-tuning, reinforcement fine-tuning and distillation. Continued pre-training is no longer among them, and it is listed for none of the models Bedrock still customises. On AWS it now runs on Amazon SageMaker AI instead, as a training job or a SageMaker HyperPod run, driven by a recipe over an Amazon Nova or open-weights model.

The volume is the other surprise. Unlabelled text has to arrive in far greater quantity than a labelled set, because each example carries so much less signal. A company that says it has “decades of documents” usually holds a fraction of what it assumed once boilerplate and repeated headers come out. Count the corpus first.

Distillation

Run a large teacher model over a representative set of your prompts, keep its responses, and fine-tune a small student model on those pairs. Amazon Bedrock Model Distillation automates both halves, generating the teacher’s responses and then fine-tuning the student. The prompts come from a JSONL file you supply, or from Bedrock invocation logs already collected in production. The result is a smaller model that reproduces the teacher’s behaviour on that task at lower inference cost and latency, and is less accurate elsewhere.

The teacher’s quality sets the ceiling exactly. A teacher that writes four flowing paragraphs when the desk wants eight fields produces a student that writes four flowing paragraphs faster and cheaper. Distillation is a cost move applied after quality is settled.

Evaluation

Side by side

Read the labelling column against the volume column and the trade sits in the open. The two jobs that need no labelling are the two that want vastly more data. The one job whose dataset a small team can build by hand is the one that needs a human judgement attached to every item. No route is light on people and light on data at the same time.

The “what it changes” column decides the rest. Only one row changes what a good answer looks like, and this desk’s complaint is entirely about what a good answer looks like. The vocabulary row would be the answer if the adjusters were reporting that the assistant mishandled trade language, and they are not, because the glossary in the prompt already closed that gap. Distillation changes neither, and the student would reproduce today’s wrong-shaped drafts.

Which job the gap calls for

Ordering the gates this way keeps the conversation honest. Capability comes first because “train our own model” collapses under one question about accelerator-months. Vocabulary comes next because it is settled by counting tokens rather than by opinion. Behaviour comes third, and it is where most teams actually land. Cost comes last, since a cheaper copy of an unsatisfactory answer is not progress.

The solution

Run an instruction tuning job on a curated set of prompt-and-completion pairs, where each prompt is a surveyor’s report plus the standing instruction, and each completion is the eight-field assessment note as an adjuster would have written it. That is a fine-tuning job on Amazon Bedrock over a dataset in Amazon S3, and it targets the one thing the adjusters are complaining about: the shape of the answer. Continuous pre-training is off the list because the vocabulary gap is already closed, and raw surveyor prose would say nothing about the eight fields even if the archive were ten times the size. Distillation is a conversation for after the drafts are good, when 22,000 claims a year makes the inference bill worth attacking.

Preparing the data

This is where the job succeeds or fails, and AWS names six elements of preparing data to fine-tune a model: data curation, governance, size, labeling, representativeness, and reinforcement learning from human feedback.

Data curation means choosing examples rather than collecting volume. The 60,000 historical notes are tempting because they exist, and most of them are unusable: notes written before the eight-field form was adopted, notes an adjuster rewrote three times, notes copied from a template with the fields left empty. Pulling all 60,000 into a training set teaches the model the average of every habit the desk has ever had, including the ones it stopped having. Select instead. Ask two senior adjusters to nominate notes they would be happy to see reproduced, and treat everything else as raw material rather than as training data.

Labelling quality and consistency come next. Two adjusters shown the same report should produce completions that agree on structure, on ordering, and on how much detail a field carries. Inconsistent labels teach the model that both versions are acceptable, and its output then lands between the two. Write a one-page rubric, have the labellers work through ten reports together before they work alone, and have a third person spot-check a sample.

Size is the number teams most often get backwards. Hundreds of clean, consistent pairs beat thousands of noisy ones, and adding noisy examples to a clean set makes the result worse rather than diluting the noise. Start at 500 to 1,000 pairs, measure, and add more only where measurement says a case is underserved. The service limits leave room for that: on the Meta Llama models the sum of training and validation records runs from 100 to 10,000, and on the Amazon Nova models it caps at 20,000, both adjustable through Service Quotas.

Representativeness asks whether the set covers the cases the model will actually meet, in something like the proportions it will meet them. A training set drawn from whatever was easiest to find will over-represent straightforward container-damage claims and under-represent the general average and total-loss cases that take an adjuster longest. The model then performs best on the work that was already quick. Build the case mix deliberately and check it against last year’s claims by category.

Governance covers where the data came from and what may be done with it. Provenance for every document, and a consent or contractual basis for using a client’s surveyor report in a training run. Personal data removed before anything leaves the bucket, with Amazon Macie finding what a manual pass misses. A retention rule for the dataset itself, and a record of which dataset version produced which custom model. This is the same care any training dataset deserves, and a training run is harder to unwind than an index, because the data is now in the weights.

Then hold back a validation split before training starts, perhaps 10 to 15 per cent, drawn to the same case mix and never shown to the model. Split by claim rather than by note, so the same claim’s near-duplicate drafts cannot land on both sides. Leakage across that boundary produces a model that scores well on the split and performs worse on the desk, and it is the failure that is hardest to spot after the fact.

Where RLHF fits

The suggestion that adjusters rank drafts describes a real technique. Reinforcement learning from human feedback (RLHF) shows people several candidate outputs for the same prompt and asks them to rank them against each other. Those rankings train a separate reward model that learns what people preferred, and the language model is then tuned to score well against that reward model. Because the reward model stands in for a human, the preference signal reaches millions of training steps without a person scoring each one.

What it teaches is helpfulness, tone and harmlessness, the qualities that are easy to recognise and hard to write down as a target output. It is how a raw pre-trained model becomes one that answers usefully rather than only plausibly, and an instruction-following model in a catalogue arrives with that work already done. It is a poor fit for this desk, where the wanted answer can be written down exactly and a labelled pair states it more directly than a ranking does. Keep ranking as the thing the adjusters do during evaluation, and keep the training on pairs.

Amazon Bedrock’s own feedback-driven method is a different technique with a similar name. Reinforcement fine-tuning generates several responses per prompt and scores them with a reward function you define, either custom code in an AWS Lambda function or a model acting as judge, then trains on those scores. No human ranks anything, and it is supported on a short list of models: Amazon Nova 2 Lite, gpt-oss-20B and Qwen3 32B. Where retrieval and a trained model end up running together, each carries a different half of the answer: retrieval supplies the facts, training supplies the form.

Worked example

Eight hundred pairs, built over three weeks by two adjusters at two days a week each. The case mix is set against last year’s claim register rather than against what was easy to export:

The last row is the one that would have been dropped by a team collecting whatever came out of the claims system first, and it is the work that costs the most adjuster time per claim.

Each pair goes into the dataset in the format the customisation job expects, one JSON object per line:

{"prompt": "Draft an assessment note from the surveyor report below. Use the eight standard fields in order, state any applicable exclusion first...\n\nSURVEYOR REPORT\n...", "completion": "APPLICABLE EXCLUSION: none identified\nPERIL: wet damage, condensation\nCAUSE: ...\n"}

That prompt-and-completion shape is what Bedrock’s non-conversational text-to-text models take, which among the models still open for customisation means Meta Llama 3.1 8B and 70B Instruct. The conversational ones, Llama 3.2 and Llama 3.3, take a Converse API record carrying system, user and assistant messages instead. Anthropic Claude 3 Haiku was the other fine-tuning target here and reached end of life on Amazon Bedrock on 10 September 2026, so leave it out of the plan. Confirm the schema for the chosen model before the labellers start.

The instruction sits in the prompt on every pair, identical each time, because that is the instruction the application will send in production and the model should learn the completion that follows it. A hundred and twenty further pairs, drawn to the same mix from claims not represented above, are held back and never trained on. They are scored before and after, by the adjusters, on whether a draft could be signed with no edits, with light edits, or not at all. The number that decides whether the run was worth doing is that first bucket, and it should be measured on the current assistant first so there is something to compare against.

What’s worth remembering

1. Pre-training builds general capability from an enormous unlabelled corpus and belongs to model providers, so “train our own model” almost always turns out to mean fine-tuning: continuing from pre-trained weights rather than random ones, which is transfer learning.
2. Fine-tuning on labelled prompt-and-completion pairs, usually as instruction tuning, changes behaviour and format, while continuous pre-training on raw unlabelled domain text changes vocabulary and teaches nothing about output shape.
3. Amazon Bedrock’s documented customisation methods are supervised fine-tuning, reinforcement fine-tuning and distillation, and continuous pre-training now runs on Amazon SageMaker AI instead.
4. Distillation trains a small student on a large teacher’s outputs, so it reduces cost and latency and reproduces whatever the teacher was already doing wrong.
5. Data curation, governance, size, labeling and representativeness decide the result more than any training setting does, and a few hundred consistent pairs outperform thousands of inconsistent ones.
6. Reinforcement learning from human feedback (RLHF) trains a reward model from human rankings and tunes the model against it, which suits helpfulness and tone rather than a task whose right answer can be written down.