Generating and Understanding Images, Audio, and Video on Bedrock

The situation

A media and operations team has landed two projects in the same sprint. Both briefs say “images”, so someone filed them under one ticket. The first is a marketing pipeline: from a product name and a short brief, produce on-brand hero images and a few seconds of promotional video, at volume, without a photoshoot. The second is a claims-intake pipeline. Customers upload PDFs, phone-camera photos of receipts, voicemails and short video clips. The business wants structured records out of all of it, so downstream systems can process a claim without anyone retyping it.

Both are “working with media on AWS”, and that is where the resemblance ends. One is generation, where the model produces the artefact. The other is understanding, where the model reads an artefact and returns structure. The tools barely overlap and the failure modes are opposite. The provenance question, can we prove which images our system made, lands on only one of them.

Solving both with “a model on Bedrock” is too coarse a plan. The first cut is which of the two jobs you are doing. Modality and output shape come after that.

What actually matters

Direction of travel decides everything downstream. Generation goes text to media: a prompt in, an image or a clip out. Understanding goes media to structure: an image, document, audio or video in, and fields, transcripts or a reasoned answer out. Name the direction first and you stop shortlisting services that solve the other problem well.

On the generation side, lifecycle status has stopped being trivia. Every entry ListFoundationModels returns carries a modelLifecycle field holding Active, Legacy or EOL. The Legacy state has concrete consequences. New customers cannot adopt a Legacy model at all, existing customers may lose access after fifteen days of inactivity, and no new Provisioned Throughput or fine-tuning job can be created against one. After the EOL date the model is removed from every Region and calls to it fail.

For models launched before 7 September 2026 the Legacy period runs at least six months. Where the EOL date falls after 1 February 2026, AWS says a model enters public extended access after a minimum of three months in Legacy, at pricing the provider sets and at which you should expect an increase. That phase has its own start date in the Legacy table, and for Nova Canvas, Nova Reel and Titan Image Generator that column is empty. Models launched since then carry an “EOL no sooner than” date on the model card plus a Legacy period of six months or forty-five days.

That policy shapes the generation half right now. No model with a published Active lifecycle turns a prompt into a still or a clip: the model-card catalogue holds two Amazon image generators and one video generator, and every one of them is Legacy or past EOL. Four prompt-to-media models do still have published inference parameters and a row in the regional availability table, all of them In-Region in us-west-2 and nowhere else, and none of them has a model card. The model card is where AWS publishes the lifecycle state and the notice period, so building on one of those four means building with no published retirement notice. The alternative for a marketing pipeline is image editing, where the Active models are, which turns the design question into how much of the work can be done by transforming an asset that already exists.

The understanding side splits three ways by the shape you need out. Structured fields and tables from mixed media at volume, through one managed pipeline: Bedrock Data Automation. Deep control over a single modality, such as query-based extraction from an awkward layout or speaker-partitioned transcription: the purpose-built services. An answer rather than a schema: a multimodal foundation model reading the file in a prompt. The two mistakes are asking a multimodal model for free-form reasoning when you needed reliable fields, and standing up three single-purpose services where one pipeline covered the mix.

What we’ll filter on

1. Direction: generating media from a prompt, or reading media into structure or an answer?
2. Lifecycle: does the model have a published lifecycle state, and is it callable from your Region at all?
3. Output shape (understanding only): strict fields and tables, single-modality precision, or free-form reasoning?
4. Breadth: one mixed stream of many media types, or one modality you want deep control over?
5. Provenance (generation only): do you need to prove later which images your system produced?
6. Operational fit: a managed pipeline with blueprints, a direct model call, or a parser wired into a Knowledge Base?

The landscape

Image generation. The catalogue lists two Amazon image generators and both are Legacy. Nova Canvas, amazon.nova-canvas-v1:0, went Legacy on 30 March 2026 and reaches EOL on 30 September 2026, in us-east-1, eu-west-1 and ap-northeast-1. Titan Image Generator G1 v2, amazon.titan-image-generator-v2:0, is already past its EOL date of 30 June 2026. Three Stability text-to-image models sit outside the model cards: Stable Image Core, stability.stable-image-core-v1:1, Stable Image Ultra, stability.stable-image-ultra-v1:1, and Stable Diffusion 3.5 Large, stability.sd3-5-large-v1:0. Each has a published request-and-response page, a row in the regional availability table showing In-Region support in us-west-2 only, and no model card, which means no published lifecycle state and no stated notice period. They are selectable, on the bare stability. id; what a pipeline on one of them does not get is a date.

Image editing. Thirteen Stability models are Active, and they are the working half of generation on Bedrock: inpaint, outpaint, erase object, search-and-replace, search-and-recolor, style guide, style transfer, control sketch, control structure, three upscalers, and remove background. Three of those make a new image rather than repairing one. Style guide extracts the stylistic elements of a reference image and follows a text prompt in that style. Control sketch follows a prompt guided by the contour lines and edges of a sketch, and control structure follows one while holding the structure of an input image.

They run only through US geo cross-region inference. The model cards mark In-Region as unsupported in us-east-1, us-east-2 and us-west-2, and give a geo inference ID such as us.stability.stable-image-style-guide-v1:0 that routes across all three. Guardrails and abuse detection apply. Converse, response streaming, Agents, Flows and Knowledge Bases do not.

Video generation. Nova Reel is the only video generator with a model card, in two versions, amazon.nova-reel-v1:0 and amazon.nova-reel-v1:1, both Legacy with an EOL date of 30 September 2026. It runs as an asynchronous job through StartAsyncInvoke and writes the MP4 to an S3 bucket you name. Luma Ray 2, luma.ray-v2:0, is the other one: a text-to-video model with the same asynchronous shape, documented through StartAsyncInvoke, GetAsyncInvoke and ListAsyncInvokes, listed In-Region in us-west-2 in the regional availability table, and absent from the model cards, so with no published lifecycle state. After 30 September it is the video generator a new build has.

Bedrock Data Automation (BDA). A managed service that takes unstructured documents, images, audio and video and returns structured insights: fields and tables from documents, transcripts and summaries from audio and video, captions, detected text and moderation labels from images. Standard output is what you get by default. Custom output comes from blueprints, which describe the fields you want for a given document or media type, so a receipt and a claim form can yield different structures through the same pipeline. Confidence scores and visual grounding come back with the extraction.

Two shapes of call exist and they are not equivalent. InvokeDataAutomation, the synchronous one, only processes images. Everything else goes through InvokeDataAutomationAsync, polled with GetDataAutomationStatus, with results written to S3. Limits are worth knowing: async documents up to 500MB and 3,000 pages with the splitter enabled, images up to 5MB, audio and video up to 240 minutes each, and 40 blueprints per project against 1,000 per account. BDA also plugs into a Bedrock Knowledge Base as a parser, but a narrower one than the API: the BEDROCK_DATA_AUTOMATION parsing strategy covers PDFs plus JPEG and PNG images, extracting figures, charts and tables as well as text, and its only setting is parsingModality: MULTIMODAL. It takes no project ARN and no blueprint, and AWS documents no audio or video path through it.

Purpose-built AI services. Amazon Textract reads documents: text detection, forms and tables through AnalyzeDocument, targeted extraction through Queries, invoices and receipts through AnalyzeExpense, identity documents through AnalyzeID. Amazon Transcribe turns speech into text in batch or streaming, partitions speakers, handles multi-channel audio and takes custom vocabulary. Amazon Rekognition analyses images and video for objects, scenes, text, faces and unsafe content. Each is deep in one lane, tunable, and long established.

Multimodal foundation models. Nova 2 Lite, Nova Lite, Nova Pro and Claude on Bedrock accept an image or a document alongside the text prompt and answer questions about it: what is wrong with this diagram, does this receipt match this policy, what is the person in this photo doing. No blueprint, no fixed schema. That suits a fuzzy or one-off question rather than a repeatable extraction job.

Evaluation

Nova Canvas, Nova Reel and Titan Image Generator are missing from the table below on purpose. All three are Legacy, which means a new project cannot adopt them, and two of them stop working entirely on 30 September 2026. The prompt-to-media models with no model card are in the table, because a new project can call them; what the table cannot show for them is a lifecycle state, since AWS publishes none.

Side by side

The solution

The marketing pipeline is a generation job, and the brand brief decides which half of it to use. For the stills, the Active path is Stability’s style guide model: give it the brand’s reference image and a text prompt per product, and it follows the look across a set. Control sketch and control structure do the same trick from a layout rather than a style. Remove background drops the result onto the clean canvas the brand guidelines require, inpaint fixes a detail, and an upscaler takes the chosen frame to campaign resolution. Every one of those is a separate model id, not a mode of one generator, so the pipeline is a short chain of calls rather than a single invoke. Stable Image Core, Stable Image Ultra and Stable Diffusion 3.5 Large would skip the reference image and generate from the prompt alone in us-west-2, which is the shorter route for a brief starting from nothing; AWS publishes no lifecycle state for any of them, so there is no notice period to plan a migration against. On-brand work needs the reference image regardless, which is why the editing family fits this brief.

Call these through the geo inference ID. In-Region invocation is not offered for the Stability editing family; the ids are prefixed us. and route across us-east-1, us-east-2 and us-west-2. A pipeline that hardcodes the bare stability. id fails on a model that is otherwise available to the account.

The clip runs on Luma Ray 2 after September. Nova Reel still works until 30 September 2026 for an account already using it, and no new project can adopt it at all. Ray 2 is the video generator a new pipeline starts on, in us-west-2 only, and it carries the same caveat as the Stability text-to-image models: no model card, so no published lifecycle state and no notice period. Design the call site so the model is swappable, because the asynchronous job shape (StartAsyncInvoke, poll GetAsyncInvoke, collect from S3) is the same for both and survives whichever model replaces them.

Provenance has to be recorded, not read back. The invisible watermark was an Amazon feature: AWS describes Nova Canvas as having built-in watermarking controls and Titan Image Generator G1 v2 as embedding one. Both models are leaving, nothing in the Stability documentation describes anything comparable, and the Bedrock user guide and API reference no longer document a way to read a watermark back, so do not design around detection. If the brand needs to answer “did our system make this?” a year from now, the pipeline answers it from its own records. Log the model id, the prompt, the reference image, the seed, the Region and a hash of the bytes as each asset is produced, and keep that ledger with the asset library. File metadata helps until the first crop-and-recompress strips it, so the ledger and the hash are the durable part.

The claims-intake pipeline is an understanding job, and the answer is BDA. The input is mixed: PDFs, photos, voicemails and clips arriving together, all needing to become records. That is the case BDA is built for, one managed API across four modalities instead of a routing layer that sniffs each upload and dispatches it. Blueprints define what “a claim form” and “a receipt” should yield, so downstream systems see the same fields every time. Making the same corpus retrievable is a second job, and BDA as the Knowledge Base parser only does part of it: that strategy covers the PDFs and the photos, extracting their figures and tables alongside the text, and AWS documents no audio or video route through it. The voicemails reach a knowledge base only by writing BDA’s transcripts back to a text data source.

When to walk away from BDA toward a purpose-built service. If the stream is actually one modality with an exacting requirement, the answer flips. Scanned forms with awkward layouts needing the strongest OCR, query-based extraction and signature detection are a Textract job. Audio needing speaker partitioning, custom vocabulary and per-channel transcription is a Transcribe job. Moderation and object detection across a video library is a Rekognition job. BDA’s advantage is breadth; the purpose-built services go deeper in one lane. Standing up all three to reconstruct what BDA does in one call wastes the breadth, and forcing a precision task through a generalist pipeline wastes the depth.

When the answer is neither. If the requirement is “read this and tell me something” rather than “extract these fields every time”, the tool is a multimodal model reasoning over the file in the prompt. Does this receipt match the policy the customer quoted? Is the damage in this photo consistent with the described incident? Nova Pro or Claude reading the image answers that in language. Pushing the same question into BDA’s structured output, or into Rekognition’s label set, gets a shape that does not fit the question.

Worked example

One web form feeds both pipelines, and two items arrive in the same minute. Marketing submits a brief: matte black insulated flask, studio lighting, plain background, in the house look. The claims team’s customer uploads a phone photo of a damaged flask, a PDF claim form and a ten-second voicemail.

The brief goes to generation. The brand’s reference image and the product prompt go to style guide, and the result goes through remove background and an upscaler. Note the model id.

brand_style = base64.b64encode(open("brand-reference.png", "rb").read()).decode()

still = bedrock_runtime.invoke_model(              # any of us-east-1/2, us-west-2
    modelId="us.stability.stable-image-style-guide-v1:0",   # geo id, not stability.*
    body=json.dumps({
        "image": brand_style,
        "prompt": "matte black insulated flask, studio lighting, plain background",
    }),
)

png = base64.b64decode(json.loads(still["body"].read())["images"][0])

The us. prefix is not optional. These models list In-Region inference as unsupported, so the bare id fails even though the account has access and the Region is right. The provenance record is written here, at the moment the bytes exist, because nothing in the output carries one.

The claims upload goes to understanding, and mixed media means BDA. The synchronous API handles images only, so a PDF, a photo and a voicemail arriving together go through the asynchronous one.

job = bda_runtime.invoke_data_automation_async(    # bedrock-data-automation-runtime
    inputConfiguration={"s3Uri": "s3://claims-intake/2026/08/claim-8812/"},
    outputConfiguration={"s3Uri": "s3://claims-structured/claim-8812/"},
    dataAutomationConfiguration={
        "dataAutomationProjectArn": CLAIMS_PROJECT_ARN,
        "stage": "LIVE",
    },
    dataAutomationProfileArn=PROFILE_ARN,
    blueprints=[{"blueprintArn": CLAIM_FORM_BLUEPRINT, "stage": "LIVE"}],
)

status = bda_runtime.get_data_automation_status(invocationArn=job["invocationArn"])
if status["status"] == "Success":
    print(status["outputConfiguration"]["s3Uri"])

The blueprint pulls claimant, policy number and itemised loss from the form. The voicemail comes back as a transcript and a summary, the photo as a caption with detected text and moderation labels. Results land in S3 rather than in the response, so a pipeline of any size makes this a step in a state machine instead of a thread in a request handler.

Point a knowledge base at the same bucket with the BEDROCK_DATA_AUTOMATION parsing strategy and the form and the photo become retrievable too, figures and tables included. The parsing strategy takes no project ARN, so the knowledge base is configured separately from the intake project, and it does not read the voicemail: for the assistant to answer “what did the customer say happened?”, the transcript BDA wrote to S3 has to be ingested as text. When a supervisor asks whether the damage in the photo matches the described incident, that is not extraction at all. The photo and the transcript go to a multimodal model, which returns a judgement in language. One intake form, three tools, chosen by direction first and output shape second.

What’s worth remembering

1. Pick the job before the model. Generation writes media from a prompt; understanding reads media into fields or an answer.
2. Legacy blocks new adoption. No new customers, no new Provisioned Throughput or fine-tuning, and existing users can lose access after fifteen idle days.
3. Amazon’s generators reach EOL. Nova Canvas and Reel go on 30 September 2026; Titan Image Generator went in June. Luma Ray 2 outlasts them.
4. Stability IDs split two ways. The thirteen Active editing models need us. geo IDs; Core, Ultra and SD3.5 Large take the bare id in us-west-2.
5. BDA’s synchronous call handles images only. Everything else goes async, with results in S3; blueprints define the output fields.
6. Depth or free-form answers: skip BDA. Use Textract, Transcribe or Rekognition for one demanding modality, a multimodal model for an answer rather than a schema.