A fast pass over the ML fundamentals and SageMaker services this track assumes as background and the AIP-C01 track above it never re-teaches.
The words at a glance
The terms nest inside one another, and most confusion comes from treating them as siblings. Artificial intelligence is the outer ring, machine learning sits inside it, deep learning inside that, and generative AI inside that again. Agentic AI is generative AI handed tools. Telling them apart is worth a slow read if the nesting is new.
| Term | What it means |
|---|---|
| Artificial intelligence (AI) | The outer ring: any system behaving intelligently, learned or coded |
| Machine learning (ML) | The part of AI that learns behaviour from data, not rules |
| Deep learning | ML built on many-layered neural networks |
| Neural network | Layers of weighted connections, trained by nudging the weights |
| Generative AI (GenAI) | Deep learning that produces content, not a label or a number |
| Agentic AI | Generative AI given tools, memory, and autonomy to plan and act |
| Large language model (LLM) | A large text model, the usual engine under generative AI |
| Computer vision | ML on images and video: detection, classification, recognition |
| Natural language processing (NLP) | ML on human language: extraction, sentiment, translation |
| Algorithm | The recipe for learning: linear regression, gradient boosting, transformers |
| Model | The artefact that recipe produces once trained |
| Training | The run that turns data plus algorithm into a model |
| Inferencing | Running the trained model on new requests: real-time, serverless, asynchronous, batch |
| Bias | A systematic skew in data or model that pushes predictions one way |
| Fairness | Whether that behaviour holds across the groups affected |
| Fit | How closely learned behaviour matches the real relationship |
Data types at a glance
| Data type | What it is | What it permits |
|---|---|---|
| Structured | Fixed schema, defined fields and types | Classic ML: regression, classification, clustering |
| Unstructured | No schema: free text, images, audio, video | Deep learning and foundation models, after embedding |
| Tabular | Rows of records, typed columns | The default for traditional ML and for Canvas |
| Time-series | Observations stamped in order, order carrying meaning | Forecasting; split by time, never shuffle |
| Image | Pixels, with meaning in shape and arrangement | Computer vision tasks |
| Text | Language, meaning depending on sequence and context | NLP and anything token-based |
| Labelled | Every example carries its answer | Supervised learning: regression and classification |
| Unlabelled | Raw examples, no answer attached | Clustering, or self-supervised pre-training |
Fundamentals at a glance
| Thing | What it is | Reach for it when |
|---|---|---|
| Regression | Labelled data, continuous target | Predicting a number: price, demand, delivery time |
| Classification | Labelled data, categorical target | Predicting a label: fraud or not, which tier |
| Clustering | Unlabelled data, grouping | Finding structure with no target |
| Semi-supervised learning | A small labelled set, a large unlabelled one | Labels are scarce, raw data is not |
| Self-supervised learning | Labels derived from the data itself | Foundation-model pre-training |
| Reinforcement learning | A reward signal from an environment | An agent learns by acting and scoring |
| RLHF | Human preference rankings train a reward model | Aligning responses to human preference |
| Transfer learning | A pre-trained model adapted with a small labelled set | You want a head start |
| Overfitting | Train loss falls, validation loss rises | Too much capacity, or too few examples |
| Underfitting | Both losses stay high | Too little capacity, too few epochs, rate too low |
| Data drift | The input distribution shifts over time | Same relationship, different inputs arriving |
| Concept drift | The input-to-target relationship shifts | The same inputs now map to a different target |
| Training-serving skew | A pipeline mismatch between training and serving | Degradation appears at once, not gradually |
| Bias and variance | High bias underfits, high variance overfits | Either can hit one group harder than the aggregate shows |
| Data collection | Gathering the raw data | The first stage, and the ceiling on every later one |
| Exploratory data analysis | Distributions, gaps, outliers | Before committing to an approach |
| Data pre-processing | Cleaning, de-duplicating, filling gaps, normalising | Raw data rarely arrives fit to train on |
| Feature engineering | Deriving the inputs the model learns from | The signal is implied, not sitting in a column |
| Model training | Running the algorithm over prepared data | The data is stable enough to justify the compute |
| Hyperparameter tuning | Searching the settings fixed before training | A trained model works and the data has more in it |
| Evaluation | Scoring against data the model never saw | Before anyone decides to ship |
| Deployment | Putting the model where requests reach it | Evaluation cleared the business bar |
| Monitoring | Watching for drift, quality, and bias | From the first real request |
| MLOps | Running that pipeline tracked, repeatable, monitored | You want DevOps rigour on models |
| Experimentation | Tracked, comparable training runs | You need to know what produced which result |
| Repeatable processes | Same data and pipeline, same model | Nobody ships a result nobody can reproduce |
| Scalable systems | Pipelines and endpoints that take more load | Real volume is coming |
| Technical debt | Bespoke glue, orphaned notebooks, unowned features | It compounds faster here, because data moves |
| Production readiness | Monitoring, rollback, ownership, documentation | Before a model carries decisions |
| Model re-training | Refreshing the model on newer data | Monitoring found drift |
Metrics at a glance
| Metric | What it measures | Watch for |
|---|---|---|
| Accuracy | Share of all predictions that were right | It reads high on a rare positive class: 99% on 1% fraud comes from always predicting “no fraud” |
| Precision | Of everything flagged, how much really was | Raise it when a false positive does more damage |
| Recall | Of everything that really was, how much got flagged | Raise it when a miss does more damage |
| F1 score | Harmonic mean of precision and recall | One number when both errors hurt |
| Cost per user | Inference and infrastructure spend per active user | It moves as adoption moves; track both |
| Development costs | People, labelling, experimentation to a first model | Paid once; weigh against a recurring saving |
| Customer feedback | Ratings, complaints, support volume, thumbs | It catches failures the metrics score as fine |
| Return on investment (ROI) | Benefit minus cost, over cost | The number the business reads; no offline metric answers it |
The SageMaker suite at a glance
Four names below are closed to new customers: Ground Truth, Augmented AI, Clarify, and Model Monitor. Existing customers carry on, and AWS keeps patching them, but no new features are coming. The jobs they name still need doing, so they stay on this sheet as vocabulary rather than as services to adopt. For monitoring, AWS points replacements at the open-source SageMaker AI monitoring solutions plus CloudWatch. The security and responsible-AI sheet carries the same note.
| Service | Job | Reach for it when |
|---|---|---|
| Ground Truth (closed to new customers) | Labels training data, using a private, vendor, or Mechanical Turk workforce | A labelled dataset is needed before training |
| Augmented AI (A2I) (closed to new customers) | Routes production predictions to a human reviewer | A live prediction needs a person first |
| Clarify (closed to new customers) | Pre-training bias metrics on the data; post-training bias, SHAP attributions and partial dependence plots on the model | You want skew measured, or a prediction explained |
| Model Monitor (closed to new customers) | Watches a live endpoint on four fronts: data quality, model quality, bias drift, feature attribution drift | You need continuous monitoring, not one check |
| Model Cards | Documentation: intended use, risk rating, training details, evaluation results | You need to record what a model is for |
| Model Registry | A versioned catalogue with approval status | You need to gate what a CI/CD pipeline promotes |
| Feature Store | One ingestion, two reads | The same features are wanted live and as training history |
| Pipelines | ML workflow orchestration | You want data-to-deployment automated |
| JumpStart | A hub of pretrained models deployed into your own account | You want a pre-built model on your own endpoint |
| Canvas | No-code model building, and the home of Data Wrangler’s visual data prep | Someone needs a model, or prepared data, without code |
| Trainium | AWS training silicon behind Trn instances; later generations also serve inference | Training a large model on cost-efficient hardware |
| Inferentia | AWS inference silicon behind Inf instances | Serving a model on cost-efficient hardware |
| Real-time inference | An always-on endpoint | Traffic is steady and latency must stay low |
| Serverless inference | 1 to 6 GB of memory, scaling to zero between requests | Traffic is intermittent, cold starts acceptable |
| Asynchronous inference | A queue in front of the endpoint: payloads to 1 GB, processing to an hour | Payloads are large or slow, and nobody waits |
| Batch transform | Scores a whole dataset offline, no endpoint | Scoring data once, with no live traffic |
Deployment strategies at a glance
| Strategy | Shape | Reach for it when |
|---|---|---|
| Shadow | A copy of live traffic hits the new version; only the production variant answers callers | You want a clean comparison, nobody exposed |
| Canary | A small share of real users hits the new version | You want to widen once the slice looks healthy |
| Blue/green | Two parallel fleets, one cutover, instant rollback | You want a whole-scale switch you can reverse |
| A/B | Traffic split deliberately to compare outcomes | You are experimenting, not rolling out |
SageMaker packages the middle two as deployment guardrails: blue/green with all-at-once, canary, or linear traffic shifting, plus rolling deployments. Shadow tests are a separate feature, and not available on serverless or asynchronous endpoints. An A/B test runs as weighted production variants behind one endpoint.
Decision rules
- If a term is confusing, place it in the nesting. AI contains ML, ML contains deep learning, deep learning contains generative AI, and agentic AI is generative AI plus tools. Almost every mix-up here treats two rings as rivals.
- If the task has one outcome that is legally or contractually right every time, write a rule, not a model. Tax bands, eligibility thresholds, and statutory retention periods need deterministic logic. A prediction that is right 99% of the time is wrong 1% of the time on something that had a defined answer.
- If the data is structured and labelled, traditional ML is usually enough. If it is text, image, or audio, you are into deep learning or a foundation model.
- If the data is time-series, split it by time. Shuffling trains the model on observations that come after the ones it has to predict, and the score looks excellent right up until production.
- If one class is rare, quote precision, recall, or F1 rather than accuracy, and say which error you would rather make.
- If training and validation error are close but one subgroup scores far worse, that is under-representation in the training data, not overfitting. More data from that group fixes it. Regularisation does not.
- If you have epochs, learning rate, or batch size, those are hyperparameters, set before training. Temperature, top-p, and top-k are inference parameters, set per request.
- Work the ML lifecycle as a loop. Business goal, collect, prepare and explore, engineer features, train, tune, evaluate, deploy, monitor, then back to collect.
- If the target is a number, that is regression. If it is a category, classification. If there is no target at all, clustering.
- If labels are scarce but raw data is not, use semi-supervised learning rather than waiting for a fully labelled set.
- If you are pre-training a foundation model, the labels come from the data itself. That is self-supervised learning, and nobody hand-labels anything.
- If you want a model to produce certain responses by human judgement, that is RLHF, a training technique rather than an evaluation method.
- If validation loss rises while training loss keeps falling, you are overfitting. Cut epochs, add data, or regularise.
- If both losses stay stubbornly high, you are underfitting. Add capacity, train longer, or raise the learning rate.
- If a model that was fine yesterday is wrong more often today and the inputs still look the same shape, suspect concept drift before the pipeline.
- If a model is wrong from the day it deployed, suspect training-serving skew. Drift takes time to appear. Skew does not.
- If training data needs labelling, that is Ground Truth’s job. If a live prediction needs a person before it acts, that is A2I’s. They sit on opposite sides of deployment.
- If you need bias measured or a prediction explained, that is Clarify’s job. If you need a live endpoint watched over time, that is Model Monitor’s.
- If you need to document what a model is for, write a Model Card. If you need to gate promotion to production, use the Model Registry’s approval status.
- If features are wanted at low-millisecond latency and also as training history, use Feature Store’s online and offline stores from one ingestion.
- If a business user needs a model with no code, point them at Canvas, not a notebook.
- If traffic is steady, use a real-time endpoint. If it is intermittent, serverless, and accept the cold starts. If payloads are large or slow, asynchronous. If there is no live traffic at all, batch transform.
- If you want nobody exposed while comparing, run a shadow test. For gradual exposure, canary. For an instant whole-scale switch, blue/green. To compare outcomes rather than roll out, A/B.
Traps
- Taking accuracy as reassurance when the positive class is rare. A model that never predicts the rare class scores in the high nineties and catches nothing.
- Reading one aggregate accuracy figure as evidence of fairness. An aggregate can stay high while one group scores badly. Subgroup analysis is what surfaces it.
- Treating inferencing as one shape. Real-time, serverless, asynchronous, and batch are four answers, and the traffic pattern picks between them.
- Confusing self-supervised with unsupervised learning. Self-supervised still trains against a target. That target is derived from the data rather than hand-labelled.
- Treating RLHF as an evaluation technique. It trains a reward model from preference data, and measures nothing after the fact.
- Assuming drift is always gradual. Training-serving skew is a pipeline bug and shows up on day one.
- Mixing up the two drifts. Data drift is the inputs changing shape. Concept drift is the mapping from inputs to target changing. Same symptom, different cause.
- Picking Ground Truth for a live prediction, or A2I for building a training set. Which side of deployment the data sits on is the clue.
- Treating a Model Card as something that blocks a release. It documents. The Model Registry’s approval status gates a pipeline.
- Assuming Clarify and Model Monitor do the same job because both mention bias. Clarify measures it in the data before training and in the model after. Model Monitor watches for it once the model is live.
- Treating Ground Truth, A2I, Clarify, and Model Monitor as services to adopt. All four are closed to new customers. Learn the distinctions they name, and expect a fresh build to assemble the same four jobs from primitives it owns.
- Reaching for a real-time endpoint out of habit when traffic is bursty and idle time is long. That is what serverless is for, cold starts and all.
- Forgetting that batch transform has no persistent endpoint. It is a different shape of job, not a cheaper real-time option.
- Treating an A/B test as a rollout mechanism. It compares outcomes. Shadow, canary, and blue/green are how you roll something out.
Say it in one line
- Labelled and continuous means regression, labelled and categorical means classification, unlabelled means clustering.
- Self-supervised learning derives its own labels from the data, which is how foundation models pre-train.
- RLHF trains a reward model from human preference rankings. It is a training step, not a measurement.
- Overfitting shows as rising validation loss against falling training loss. Underfitting shows as both staying high.
- Data drift is the inputs changing, concept drift is the input-to-target mapping changing, and training-serving skew is a pipeline bug that bites from day one.
- Ground Truth labels training data. A2I reviews production predictions. Opposite sides of deployment.
- Clarify measures bias and explains predictions. Model Monitor watches a live endpoint over time. Both services are closed to new customers.
- Model Cards document. The Model Registry’s approval status gates what ships.
- Feature Store serves the same features online at low-millisecond latency and offline as training history, from one ingestion.
- Real-time is always-on, serverless scales to zero with cold starts, asynchronous queues large payloads, and batch transform scores offline with no endpoint.
- Hyperparameters are set before training. Inference parameters are set per request.
- Shadow exposes nobody, canary exposes a slice, blue/green cuts over wholesale, and A/B is an experiment rather than a rollout.
- Accuracy, precision, recall, and F1 judge the model. Cost per user, development costs, customer feedback, and ROI judge whether it was worth building.
- MLOps is experimentation, repeatable processes, scalable systems, managed technical debt, production readiness, monitoring, and re-training.