The situation
A produce-box delivery business has around forty thousand subscribers across four cities and six years of trading behind it. That history sits in three places: a deliveries table with every box packed and dispatched since launch, a support inbox with about ninety thousand messages in it, and a subscriptions table recording every pause, resume, substitution and cancellation.
Six departments have turned up in the same fortnight, each with an ask.
- Operations: how many boxes will we need next Tuesday?
- Retention: will this subscriber cancel this month?
- Support: which of our eleven queues does this incoming message belong to?
- Marketing: are there natural groups among our subscribers that we should talk to separately?
- Finance: is this refund request unlike anything we normally see?
- Product: what should we suggest this subscriber adds to their box?
None of these is a technology question yet. Each is a business question, and it has to be turned into a machine-learning problem of a recognised type before anybody opens a console. Get that translation right and the service choice is nearly mechanical. Get it wrong and you spend a quarter building a well-engineered answer to a question nobody asked.
What actually matters
Start with the shape of the answer, because it does more filtering than anything else. Across the six asks there are five shapes: a number, a label drawn from a fixed set, a group with no fixed set and no names, a score saying how unusual something is, and a ranked list. Ask a department to say out loud what a good answer would look like written on a screen, and the shape falls out of their own words. “About 1,850 boxes” is a number. “Billing” is a label. “These four thousand subscribers behave alike” is a group. “This one scores 0.93 for strangeness” is a score. “Try the sourdough, the free-range eggs, the coriander” is a ranked list.
The second question is whether historical examples of the right answer exist. Retention has them: six years of subscriptions where each account either cancelled or did not, so every past subscriber carries the answer as a fact. Support has them too, because every one of those ninety thousand messages was filed into a queue by a human, and that filing is a label. Marketing does not have them, because nobody has ever written down which group a subscriber belongs to; the groups do not exist yet. Where labelled examples exist the model learns to reproduce them, which is supervised learning. Where they do not, the model has to find structure on its own, which is unsupervised learning. This single question splits the six asks into two families before any service is named.
The third question is whether the target is ordered in time. Predicting box demand and predicting a subscriber’s lifetime value both produce a number, so both look like regression. They are not the same problem. Demand next Tuesday depends on demand last Tuesday, on the Tuesday before that, on the school holidays and on whether the previous week was wet. Those dependencies only make sense in sequence. A target with a timestamp on it and a history that must be read in order is forecasting, and the modelling, the validation split and the evaluation all change accordingly. A validation set chosen at random from a time series leaks the future into the training data.
The fourth question is whether the answer has to differ per subscriber. Support queue routing gives every message the same treatment: the same eleven labels, the same model, no notion of who is asking. The box suggestion is the opposite, since a good answer for one household is a bad answer for the next, and the model has to hold each subscriber’s own history. That is a different technique family and, on AWS, a different service.
There is a fifth question that stops some asks from becoming machine-learning problems at all. If the answer is a definite value that a rule can compute, then a prediction is the wrong tool: a subscriber who has missed three payments is delinquent by definition, not by estimate, and a query answers that better than a model does. A boring baseline is the honest comparison for every one of these six, and at least one of them usually loses to it.
What we’ll filter on
- Shape of the output: a number, a label from a fixed set, a group with no fixed set, an unusualness score, or a ranked list.
- Labelled examples: does the history contain the right answer for past cases, recorded by somebody at the time?
- A fixed set of categories: are the possible answers known and enumerable in advance, or does the model have to discover them?
- Time order: does the target carry a timestamp, and does the history have to be read in sequence for the answer to make sense?
- Personalisation: is there one answer for everybody, or a different answer per subscriber?
The landscape
Six problem types cover almost everything a business of this size will ask for, and they are the vocabulary the AI Practitioner material uses throughout.
Regression
Predicts a continuous number from a set of input features. Lifetime value, expected weight of a box, days until the next order. It needs labelled examples, which for regression means past rows where the number actually turned out to be something and somebody recorded it. Accuracy is reported as an error distance rather than a percentage right, because a prediction of 1,820 against an actual 1,850 is neither correct nor wrong.
Classification
Predicts a label from a fixed, known set. When the set has two members (cancels or does not, fraudulent or not) it is binary classification; with more than two (eleven support queues) it is multi-class classification. It needs labelled examples and it needs the categories decided in advance, because a classifier can only ever return a label it was trained on. Accuracy, precision, recall and F1 all apply here.
Clustering
Groups records by similarity when nobody has labelled anything. It is unsupervised: there is no right answer to reproduce, so there is no accuracy score, and two runs with different settings can produce equally defensible groupings. The output is group membership plus whatever a human reads into the groups afterwards. Naming the segments is a job for a person looking at the cluster centres, not for the algorithm.
Anomaly detection
Scores how far a record sits from the pattern of everything else, which suits situations where the normal cases are plentiful and the interesting ones are rare and not necessarily alike. It runs unsupervised, so it does not need a pile of confirmed-bad examples to get going. That is why it gets reached for when a business has thousands of ordinary refunds and a handful of odd ones nobody has ever tagged.
Forecasting
Predicts numbers at future points in time from a history read in order. It is regression with time as the structure, plus the things time brings with it: seasonality, trend, holidays, and related series such as weather or promotions that move the target. The validation split has to respect the clock, holding out the most recent period rather than a random sample.
Recommendation
Produces a ranked list of items for one particular person, learned from interaction history: what each subscriber has bought, added, skipped and returned. The training data is the interaction log, the answer is personalised, and the evaluation asks whether the items a subscriber actually chose appeared near the top of the list.
The application families worth recognising by name
Alongside the problem types, the AI Practitioner material expects a handful of real-world application families and the AWS service most associated with each. Computer vision covers anything that reads an image or a video, and Amazon Rekognition is the managed front door for labels, faces and moderation. Natural language processing (NLP) covers reading and understanding text, and Amazon Comprehend does sentiment, entities, key phrases and custom text classification without a model being trained from scratch. Speech recognition is turning audio into text, which is Amazon Transcribe. Recommendation systems are Amazon Personalize. Fraud detection is a classification or anomaly-detection problem applied to transactions. Forecasting is demand, staffing and inventory planning over time. Knowledge bases are retrieval over a corpus of documents so that a foundation model answers from your own content. On AWS that is Amazon Bedrock Knowledge Bases. And agentic AI is a model given tools and a goal, deciding for itself which calls to make in what order to complete a multi-step task.
The families are not techniques. They are the shapes businesses recognise, and each one resolves down to one or more of the six problem types once you ask what the answer looks like. Knowing which family an ask belongs to is often enough to decide whether a managed service already does the job, which is the case more often than teams expect.
Evaluation
Side by side
| Technique | Output shape | Labelled examples | Fixed set of categories | Time order matters | Personalised per subscriber |
|---|---|---|---|---|---|
| Regression | A number | ✓ | ✗ | ✗ | ✗ |
| Binary classification | One of two labels | ✓ | ✓ | ✗ | ✗ |
| Multi-class classification | One of many labels | ✓ | ✓ | ✗ | ✗ |
| Clustering | Group membership | ✗ | ✗ | ✗ | ✗ |
| Anomaly detection | An unusualness score | ✗ | ✗ | ✗ | ✗ |
| Forecasting | Numbers at future times | ✓ | ✗ | ✓ | ✗ |
| Recommendation | A ranked list of items | ✓ | ✗ | ✓ | ✓ |
Two columns do most of the separating. The labelled-examples column splits supervised work from unsupervised, and the output-shape column splits everything else. The remaining three break the ties: fixed categories separates classification from clustering, time order separates forecasting from plain regression, and personalisation separates recommendation from everything above it.
From answer shape to technique
The six asks, mapped
| Ask | Answer shape | Problem type | Where it lands |
|---|---|---|---|
| How many boxes next Tuesday? | A number per future day | Forecasting | Amazon SageMaker AI |
| Will this subscriber cancel this month? | One of two labels | Binary classification | Amazon SageMaker AI |
| Which of eleven queues? | One of eleven labels | Multi-class classification | Amazon Comprehend custom classification |
| Are there natural subscriber groups? | Group membership | Clustering | Amazon SageMaker AI |
| Is this refund request unusual? | An unusualness score | Anomaly detection | Amazon SageMaker AI |
| What should we suggest they add? | A ranked list of items | Recommendation | Amazon Personalize |
Two services that used to appear in this table have gone. Amazon Forecast and Amazon Fraud Detector are both closed to new customers, and neither appears on the AI Practitioner in-scope service list. Forecasting and fraud detection are now built on Amazon SageMaker AI or on a foundation model, so neither service is a selectable landing place any more.
The solution
Operations gets a forecast. Six years of daily dispatch counts per city is a time series. The target is a number at a future date, and the useful signal lives in the sequence: weekly rhythm, school terms, public holidays, the slump in the second week of January. Train on everything up to a cut-off and validate on the most recent eight weeks. Hold the model to an error measured in boxes rather than a percentage correct, because Operations plans in boxes.
Retention gets binary classification. Every account in the subscriptions table has already resolved to cancelled or still active, and that resolution is the label. The features are the things visible before the decision: pauses, substitutions, complaint tickets, weeks since the last change. Precision and recall matter more than accuracy here. Most subscribers stay, so a model that predicts “will not cancel” for everybody scores well on accuracy and is worth nothing.
Support gets multi-class classification over text, and this one does not need a bespoke model built from first principles. Ninety thousand messages already filed into eleven queues is a labelled text dataset, and Amazon Comprehend trains a custom classifier on it directly. It is an NLP problem with a purpose-built managed service sitting exactly on top of it.
Marketing gets clustering, with a caveat. There are no labels, so the model finds groups by similarity in the features it is given. The number of groups is a setting somebody chooses rather than an answer the data supplies. Marketing then has to look at what came back and decide whether the groups mean anything. If they arrive already knowing the four segments they want and can point at examples of each, they do not have a clustering problem; they have a classification problem with the labelling work still to do.
Finance gets anomaly detection. There is no meaningful pile of confirmed-fraudulent refunds to train on, and the interesting cases are rare and unlike each other, which rules out treating it as classification today. Score every request for distance from the ordinary pattern, route the high scorers to a human, and record what that human decides. After a year of those decisions the business has labelled examples and can revisit the problem as classification, which is the usual path fraud detection takes.
Product gets recommendation. The interaction history is already there in the deliveries table: every add, skip, substitution and return. Amazon Personalize takes that log directly, and the answer is personalised by construction rather than by a rule bolted on afterwards.
Two mistakes account for most wrong turns here. The first is treating clustering as classification with the labels missing. They produce different things: a classifier reproduces a decision somebody already knows how to make, and clustering proposes a structure nobody has named. If the business can describe the categories, the work is labelling data, not clustering it. The second is treating a demand forecast as a fresh technique to be learned. It is regression with time as the structure. Everything unfamiliar about it comes from the clock: features built from lags and calendars, a validation split that respects order, and evaluation over a horizon rather than a single row. The learning-problem taxonomy sits underneath all six of these, and every one of them resolves through it.
What’s worth remembering
- Ask what a good answer looks like written on a screen: a number is regression, a label from a fixed set is classification, and a grouping with no fixed set is clustering.
- Labelled examples in the history mean supervised learning is available; their absence pushes the work towards clustering or anomaly detection, and towards collecting labels so the problem can be revisited later.
- A target with a timestamp whose history has to be read in order is forecasting, which is regression with time as the structure rather than a separate technique.
- An answer that has to differ per person is a recommendation problem and lands on Amazon Personalize, not on a classifier with the subscriber identifier added as a feature.
- Learn the application families by name, because computer vision maps to Amazon Rekognition, natural language processing (NLP) to Amazon Comprehend, speech recognition to Amazon Transcribe, recommendation systems to Amazon Personalize, and knowledge bases to Amazon Bedrock Knowledge Bases.
- Amazon Forecast and Amazon Fraud Detector are closed to new customers and off the in-scope service list, so forecasting and fraud detection are built on Amazon SageMaker AI or a foundation model instead.