The situation
A produce-box delivery business has forty thousand subscribers across four cities and six years of trading behind it. A new analytics hire is handed a blunt question by the finance director: of the things we already store, which could become a model this quarter, and what would each one cost? The labelling budget for the quarter is zero.
Four datasets are on the table.
- Subscriptions. A table in Amazon Redshift, one row per account: sign-up date, city, box size, pauses, substitutions, complaints raised, and a
cancelledcolumn that is true for about eleven thousand of them. - Delivery volumes. Three years of daily dispatch counts per city. One row per city per day, around four thousand four hundred rows in total, each with a date and a number.
- Delivery notes. Roughly ninety thousand scanned paper notes sitting in Amazon S3, one JPEG each, most with a driver’s handwriting somewhere in the margin. Nobody has typed any of them up.
- Support email. Four hundred thousand messages exported to S3 as plain text. They have never been filed, tagged, or routed into queues; the inbox was worked top to bottom and then archived.
Two of these can be turned into a working model without anybody buying a single label. One needs a small purchase first. One is not a machine-learning problem yet at all. Which is which comes out of the data, not out of a service catalogue.
What actually matters
Start with the form the data arrives in. Structured data is already carved into fields with agreed meanings: rows and columns, a schema, one type per column, a name saying what each column holds. Unstructured data has none of that. A scan is a grid of pixels. An email is a run of characters. Any structure inside either has to be extracted before a traditional model can read it. The AI Practitioner material lists the types it expects you to recognise in a single line, in its own spelling: “labeled and unlabeled, tabular, time-series, image, text, structured and unstructured”. Tabular is the ordinary rows-and-columns case, and the subscriptions table is one. Time-series is tabular with an ordering constraint bolted on. The rows are a sequence, each row’s position on the clock is part of what it means, and shuffling them destroys information. Image and text are the two unstructured cases here. Both need a step before training rather than a step during it.
Then labels, which decide more than anything else on this list. A label is the answer recorded next to an example: the category this message belonged to, the number this day actually turned out to be, the words written on this scan. Labels are either found or bought. Found labels are the ones somebody recorded for a different reason and left behind, which is why the cancelled column is worth more than it looks. Bought labels come from people reading examples and typing answers, and Amazon SageMaker Ground Truth is the AWS service for organising that work, whether it goes to your own staff, a vendor workforce, or Amazon Mechanical Turk. Purchase is a real cost with a real unit price. A two-way category on a short email is cheap; a transcription of handwriting is not. Ninety thousand scans and four hundred thousand emails are therefore very different asks, and the cost of labels, more than the merits of any algorithm, settles which method a team can afford this quarter.
Labels then decide the learning method, and the mapping is close to mechanical. Every example carrying an answer allows supervised learning, where the model is trained to reproduce those answers on cases it has not seen. No example carrying an answer allows unsupervised learning, where the model looks for structure without a target to hit, which is what clustering and anomaly detection do. A small labelled set beside a large unlabelled one allows semi-supervised learning, where the labelled part teaches the model enough to make provisional guesses about the rest and a human reviews the ones it is least sure of. Answers generated from the data itself, with no person involved, allow self-supervised learning. Hide a word in a sentence, train the model to predict it, and every sentence you own becomes training data. That is how foundation models are pre-trained. It is also why a model that has never seen your data can still read your email.
One method sits outside all of this because it does not start from a dataset. Reinforcement learning needs an environment: something that receives an action, changes state, and returns a score. An agent tries actions, collects rewards, and adjusts towards the behaviour that scores best over time. A static export of last year cannot do that, because it never responds. The variant worth knowing by name is reinforcement learning from human feedback (RLHF). People rank a model’s candidate outputs against each other, those rankings train a separate reward model, and the reward model supplies the score during fine-tuning. It is how a raw pre-trained model becomes one that answers helpfully rather than only plausibly.
What we’ll filter on
- Structured or unstructured: does the data arrive in fields with agreed meanings, or does structure have to be pulled out of it first?
- A target column: is there a value already sitting in the data that a model would be asked to reproduce for new cases?
- Time order: are the rows a sequence, where a row’s place on the clock is part of its meaning?
- Label coverage: how many examples carry an answer somebody recorded, and what would the remainder cost to label?
- An environment that scores actions: is there something that responds to a decision with a reward, or only a record of what already happened?
The landscape
The five learning types below are the whole vocabulary this scenario needs. They are usually taught as five techniques, which hides how much of the choice is made for you by what the data already looks like.
Supervised learning
Every training example carries its answer, and the model learns the relationship between the input columns and that answer. Two shapes cover most of it: a continuous number is regression, a label from a fixed set is classification. Evaluation is straightforward, because there is a right answer to compare against. Accuracy, precision, recall, and error distances all mean something. Tabular data with a target column is the classic home for this, and Amazon SageMaker AI is where it lands on AWS. Labelled text has a shortcut, since Amazon Comprehend trains a custom classifier directly from examples.
Unsupervised learning
Nothing carries an answer, so there is nothing to reproduce. The model reports structure it finds: which records resemble each other (clustering), which sit far from the pattern of the rest (anomaly detection), which themes recur across a body of documents (topic modelling). There is no accuracy score, because there is nothing to be accurate against. Two sensible runs can produce two different, equally defensible groupings, and the output is a proposal a human has to read and name.
Semi-supervised learning
A small labelled set plus a large unlabelled one, which describes most real datasets once anyone looks. Label a couple of thousand examples properly and train on those. Run the result across the rest, keep the confident predictions as provisional labels, and send the uncertain ones to a person. Ground Truth’s automated labelling works this way, which is why labelling a corpus rarely costs the full corpus multiplied by the unit rate.
Self-supervised learning
The labels come from the data itself. Mask a word and predict it, take the first half of a sentence and predict the second, corrupt an image and reconstruct it. No person labels anything, yet the model still trains against a target with a loss function. That is the part people get wrong: this is not unsupervised learning under a friendlier name. Foundation-model pre-training runs this way at enormous scale, which is why Amazon Bedrock can hand you a model that already reads English without your having contributed a single label.
Reinforcement learning
An agent, an environment, actions, and a reward. No labelled examples exist; the environment scores what the agent does and the agent optimises for that score over many attempts. It suits problems where the right answer is not known in advance but a good outcome is recognisable when it happens: routing, pricing, game play, robot control. RLHF applies the same machinery to language models. A reward model stands in for the environment, so human preference reaches millions of training steps without a person present at each one.
Evaluation
Side by side
| Dataset | Data type | Structured | Labelled | Method available now | Where it lands on AWS |
|---|---|---|---|---|---|
| Subscriptions table | Tabular | ✓ | ✓ (found: cancelled) |
Supervised (classification) | Amazon Redshift to Amazon SageMaker AI |
| Daily delivery volumes | Time-series | ✓ | ✓ (the count itself) | Supervised (forecasting) | Amazon SageMaker AI |
| Scanned delivery notes | Image | ✗ | ✗ | None until extraction, then semi-supervised | Amazon S3 to Amazon Textract |
| Support email | Text | ✗ | ✗ | Unsupervised, or a pre-trained model | Amazon S3 to Amazon Comprehend or Amazon Bedrock |
Which method the data allows
The solution
The subscriptions table is ready today. It is tabular, structured, and labelled by accident: nobody set out to build a training set, but every account has resolved to cancelled or still active, and that resolution is a label. Around eleven thousand positives against twenty-nine thousand negatives is a workable balance, and supervised binary classification is available with no labelling spend at all. Two cautions come with found labels. The first is leakage. A cancellation_date or final_refund_issued column is a consequence of the answer rather than a predictor of it, and leaving it in produces a model that scores beautifully in testing and predicts nothing in production. The second is that accuracy is the wrong headline number when most subscribers stay. Always answering “will not cancel” scores seventy-three per cent and helps nobody. Redshift can train the model in place, or the table can be exported to S3 and trained in Amazon SageMaker AI.
Delivery volumes are ready too, with one constraint. Every past day carries the number that day actually turned out to be, so this is supervised learning, and again nobody had to label anything. What makes it time-series rather than plain tabular is that the rows are a sequence. Tuesday’s count depends on last Tuesday’s, on the school holidays, and on whether the previous week was wet. Treat it as ordinary tabular data and you will split it at random into training and test sets. That puts next March in the training data and last February in the test data, and reports an accuracy nobody could reach in production. Hold out the most recent weeks instead, train on everything before them, and measure the error over that held-out period. That single ordering constraint is the whole difference between a time-series problem and a tabular one, and it is where the translation from business question to problem type most often goes wrong.
The delivery notes are not a machine-learning problem yet. They are unstructured image data with no labels. No supervised method is available, and clustering raw scans would tell you nothing anyone wants to know. The first move is extraction rather than learning. Amazon Textract reads a scan and returns text, key-value pairs, and table cells with no training and no labelled examples, which turns an image into structured fields. Once the notes are fields, the ordinary tabular options open up. If a model is still wanted afterwards, say one that flags notes carrying a handwritten complaint, that is a small labelling job: two thousand examples through SageMaker Ground Truth, a first model trained on those, then semi-supervised expansion across the remaining eighty-eight thousand with a person checking the low-confidence cases.
The support email has two routes, and they cost very differently. Unsupervised learning works on it today: cluster the messages or run topic modelling to find out what people actually write about, and use Amazon Comprehend to pull entities, key phrases, and sentiment without training anything. That produces understanding rather than a classifier. If a classifier is what is wanted, the cheap route no longer starts with labelling. A foundation model has already been pre-trained by self-supervised learning on enormous quantities of text, and it can sort messages into named categories from the category descriptions alone. Whether that is the right call or a custom model is a separate judgement about cost per message, explainability, and volume: at four hundred thousand messages a month, labelling two thousand and training a Comprehend custom classifier is often cheaper to run than calling a foundation model on every one. Either way, a simple baseline over bag-of-words features deserves to be beaten before anything larger is bought.
Reinforcement learning is absent from all four, and for the same reason each time. Every one of these datasets is a record of what already happened, and none of them responds to a decision. The substitution engine could become a reinforcement learning problem. That needs somebody to start capturing whether each substitution was accepted or refunded and to feed it back as a reward, which is a change to the product rather than to the modelling. The common mistake is reaching for reinforcement learning because a problem sounds like a sequence of decisions, with nothing available to score them. Before any of this data reaches a model, the checks on what the corpus contains matter as much for support email as for anything else, since four hundred thousand customer messages carry names, addresses, and card fragments that were never meant to leave the inbox.
What’s worth remembering
- Structured data arrives in fields with agreed meanings, so tabular and time-series data can be trained on directly while image and text need an extraction step first.
- Labels decide the method: labelled data allows supervised learning, unlabelled data allows unsupervised learning, and a small labelled set beside a large unlabelled one allows semi-supervised learning.
- Self-supervised learning is not unsupervised learning, because it still trains against a target derived automatically from the data rather than written by a person, and that is how foundation models pre-train.
- Time-series data is tabular with an ordering constraint, which makes a random train/test split wrong; hold out the most recent period instead.
- Reinforcement learning needs an environment that scores each action rather than a stored dataset, and RLHF is the variant where human preference rankings train a reward model that supplies the score.
- Look for labels you already own before budgeting for any, because a cancelled flag, a filing decision, or yesterday’s actual count is a label somebody recorded for another reason. What the remainder would cost through SageMaker Ground Truth usually decides the method.