A fast pass over what a generative model actually is before any AWS service wraps it: the architectures, the vocabulary, the limitations, and the prompting techniques the rest of the track assumes you already have.
Architectures at a glance
| Architecture | How it works | What it is for |
|---|---|---|
| Transformer | Self-attention across the whole input at once | Large language models: text generation, reasoning, chat |
| Diffusion | Iterative denoising from random noise toward a target | Image generation |
| VAE | Encode into a probabilistic latent space, then decode samples | Generating new examples that resemble the training data |
| GAN | A generator and a discriminator trained against each other | Realistic images and synthetic data |
Vocabulary and limitations at a glance
| Term | What it means |
|---|---|
| Embedding | A semantic vector representation of text, image, or audio |
| Token | The unit a model reads and generates in; cost and limits are counted in tokens, not words |
| Context window | The maximum tokens a model can hold in one call; the binding constraint on long documents |
| Multimodal | A model that takes in or produces more than one type of content: text, image, audio, video |
| Hallucination | A fluent, confident answer that is not supported by fact or by the given context |
| Knowledge cutoff | The point past which a model has no training data and so no knowledge of events |
| Nondeterminism | The same prompt can produce a different answer on different runs |
| Bias | Systematic favouritism or disadvantage reflected in training data or output |
| Toxicity | Harmful, offensive, or abusive content in a model’s output |
| Context-window limit | Content beyond the window is simply unavailable to the model, not summarised for free |
Prompting techniques at a glance
| Technique | What it does | Use when |
|---|---|---|
| Zero-shot | Instruction only, no examples | The task is common and well understood by the model |
| Few-shot | Instruction plus a handful of examples | Output format or edge cases need pinning down |
| Chain-of-thought | Asks the model to reason step by step | Multi-step logic, maths, or layered decisions |
| Self-consistency | Samples multiple reasoning paths and takes the majority answer | A single chain-of-thought pass is not reliable enough to trust |
| Negative prompt | States what to avoid rather than what to produce | Mostly image generation: no text, no watermark, no extra limbs |
| Prompt chaining | The output of one prompt becomes the input to the next | A task naturally breaks into smaller stages |
Decision rules
- If the task is generating or reasoning over text, that is a transformer; if it is generating an image from noise, that is diffusion.
- If a description mentions encoding into a probabilistic latent space and decoding samples from it, that is a VAE; none of the other three works that way.
- If two networks are described as trained against each other, generator versus discriminator, that is a GAN.
- If cost or a length limit is being discussed, think in tokens, not words or characters.
- If a document will not fit in one call, the binding constraint is the context window, and content past it is not seen at all.
- If an answer is fluent but unsupported by the given source, that is hallucination; if it is stale rather than wrong, that is the knowledge cutoff; the two are different failures with different fixes.
- If the same prompt gives different answers on repeated runs, that is nondeterminism, and lowering temperature reduces but does not eliminate it.
- If the task is common and the model already handles it, use zero-shot and do not spend tokens on examples.
- If a single chain-of-thought pass is not trustworthy enough, sample several and take the majority answer with self-consistency, at the cost of more tokens.
- If an image keeps including something unwanted, state it in a negative prompt rather than trying to word the positive prompt around it.
- If a task naturally has stages, chain prompts rather than asking for everything in one pass.
Traps
- Describing a transformer as “the architecture behind image generation.” Diffusion is the image architecture; transformers are the language one, though the two are sometimes combined in practice.
- Missing the VAE’s fingerprint phrase. If a description mentions a probabilistic latent space and decoding samples from it, that is a VAE, not a GAN or a diffusion model.
- Treating embeddings as a model output you read directly. They are a vector representation for comparison and search, not a human-readable answer.
- Assuming a longer context window fixes hallucination. It only changes what fits in the prompt; a model can still hallucinate about content that is right there in the context.
- Confusing hallucination with a knowledge-cutoff gap. Hallucination is a confident, wrong answer; a cutoff gap is the model correctly having no idea, though it can still hallucinate rather than admit that.
- Treating temperature zero as a guarantee of identical output every time. It reduces variance sharply but is not an absolute determinism guarantee across all settings and infrastructure.
- Piling on more few-shot examples to fix a reasoning failure. Examples fix format and edge cases, not multi-step logic; that is what chain-of-thought and self-consistency are for.
- Using a negative prompt on a text task and expecting the image-generation effect. Diffusion pipelines take a negative prompt as a real parameter; in a text prompt it is just another instruction the model may or may not follow.
Say it in one line
- Transformers generate language through self-attention; diffusion generates images through iterative denoising.
- A VAE encodes into a probabilistic latent space and decodes samples from it; a GAN pits a generator against a discriminator.
- Tokens, not words, are what cost and context-window limits are measured in.
- The context window is the binding constraint on long documents; content past it is unavailable, not summarised for free.
- Hallucination is a confident wrong answer; a knowledge cutoff is a correct gap in what the model has ever seen.
- Nondeterminism means the same prompt can answer differently on different runs; low temperature reduces it but does not remove it.
- Bias and toxicity are two of the named generative-AI limitations, alongside hallucination, knowledge cutoff, nondeterminism, and context-window limits.
- Zero-shot needs no examples; few-shot fixes format and edge cases, not reasoning.
- Self-consistency samples multiple reasoning paths and takes the majority vote, trading tokens for reliability.
- Negative prompts state what to avoid, mainly for image generation; prompt chaining breaks a task into stages, feeding one prompt’s output into the next.