The situation
An online retailer sells around 60,000 product lines across homewares, groceries and small appliances. About 19,000 of them carry a description somebody wrote. The other 41,000 show whatever the supplier sent, usually a model number and a spec table. Those pages convert at roughly half the rate of the ones with prose on them.
Three generative features came out of the same planning session. The first writes a description for every line that lacks one, from the supplier’s spec sheet. The second generates 400 lifestyle images for a spring homewares campaign, instead of booking a photographer and a studio. The third puts a two-sentence summary of customer reviews at the top of each product page. All three worked in the prototype, and the demo went well enough that a launch date got written down.
Then legal read it. The memo that came back does not say no on principle. It lists five things that have to be answered before anyone signs, and it is written in the vocabulary of exposure rather than the vocabulary of models. Who is liable if a generated sentence turns out to belong to somebody else. What happens when a shopper acts on a claim the model invented. What the retailer says when a customer works out that the review summary was never read by a person.
What actually matters
The AI Practitioner material files this under the legal risks of working with generative AI, and names five of them. Intellectual property infringement claims, biased model outputs, loss of customer trust, end user risk, and hallucinations. They arrive together in a memo, but they are five different problems, and the useful move is to work out where each one actually closes before arguing about which model to use.
Intellectual property infringement claims run in two directions. There is what goes in, which here means prompting the model with a competitor’s product copy or fine-tuning on a scraped corpus of descriptions the retailer has no right to. And there is what comes out. Generated text can reproduce wording that belongs to someone else, and a generated image can reproduce a protected character, a distinctive brand element, or a photographer’s recognisable work. The first direction is entirely within the retailer’s control and is a matter of what the team is allowed to paste into a prompt. The second is a property of the model and its training data. It is also the one place in this whole memo where a supplier’s contract can carry some of the liability instead of the retailer.
Biased model outputs show up here in a form that surprises people, because nobody wrote a rule and no protected attribute appears in the input. Run the generator over the whole catalogue and read the results by category. The cleaning products and the kitchen storage come back addressed to a harried mother. The power tools and the audio gear come back as flat technical specification. The model learned that association from its training corpus and applied it consistently, at a scale no copywriter would have. The image generator does the same thing in pictures: 400 lifestyle shots where the person in the kitchen is a woman and the person in the shed is a man. The exposure is a discrimination complaint, and the reputational damage arrives first.
Loss of customer trust attaches to the review summary more than to anything else. The summary sits above the reviews in the retailer’s own voice, and a shopper reading it reasonably assumes an editor read the reviews. The harm comes from the discovery rather than from the text. A summary that is accurate, useful and quietly machine-written is still a story once a customer finds out sideways. Sideways is how they find out when a competitor, a journalist or a detection tool gets there first.
End user risk is the one where somebody outside the company gets hurt. A shopper reads “suitable for induction hobs” on a pan that is not. Or “contains no nuts” on a product whose supplier sheet never said so. Or an outdoor rating on a heater built for indoor use. The retailer’s normal remedy for a bad page is to correct it, and correction arrives after the burn, the reaction, or the fire. This risk is not proportional to how many outputs are wrong. One wrong allergen line across 41,000 correct descriptions is the whole exposure.
Hallucinations are the mechanism underneath most of that. A model asked to write a description from a thin spec sheet produces a fluent one either way. The missing capacity, the invented warranty length and the confidently stated dishwasher-safe all come out in the same register as the true parts of the paragraph. Nothing in the output marks the invented clause. That makes hallucinations an architectural problem rather than a wording problem: either the model is given the facts and constrained to them, or it is not permitted to assert them.
Sorting those five, three closure classes fall out. One risk moves by choosing a different model and reading the contract that comes with it. Three are engineering problems, solvable in the pipeline by grounding, filtering and marking. And two of them, in part, come down to a person deciding, because no automated check can be trusted with the sentence that tells a shopper what is safe to eat. Notice what is missing from that list: none of the five gets closed by the model simply being better. A more capable model writes more convincing wrong allergen lines.
What we’ll filter on
- Where the liability sits: does any party outside the retailer carry part of it, and under what conditions?
- Licensing and acceptable use: what do the model’s terms permit for commercial output, and do they restrict the use we have in mind?
- Traceability: can every factual claim in the output be pointed back to a record the retailer owns?
- Automatic refusal: can the system decline to publish an output that fails a check, with nobody in the loop?
- Provenance: can we prove later that a given image or paragraph came out of a model, and which one?
- Cost of the human step: how many outputs need a person to read them, and is that number small enough to staff every week?
The landscape
Choosing the model, and reading what arrives with it
Model access in Amazon Bedrock starts in the model catalogue. Each model is requested per account, and access is granted only once the provider’s terms are accepted. Two documents matter and they are not the same document. The licence, or end user licence agreement, sets out what may be done with the model and its output. The acceptable use policy sets out what may not be generated at all. Both are provider-specific, both are linked from the catalogue entry, and clicking through them at access-request time is where most teams accept terms nobody has read.
Licensing separates hosted proprietary models from open-weight ones sharply. A model deployed from Amazon SageMaker JumpStart runs on the retailer’s own endpoint, which feels like ownership and is not. Open weights commonly arrive under a community licence with named prohibited uses, attribution requirements, and an obligation to pass the same terms to anyone you redistribute to. Some carry a user-count threshold above which the licence stops applying at all. Running the model in your own account changes none of that. It also means there is no provider standing behind what comes out: an open-weight model gives you the weights and hands you the output risk with them.
Against that, some providers indemnify customers against third-party copyright claims arising from generated output, and AWS extends uncapped intellectual property indemnity covering a defined list of models available through Bedrock. The conditions are the part worth memorising. The protection applies to the listed models, it depends on the customer not disabling the service’s filters, and it does not survive somebody deliberately prompting for infringing content. It is a list with conditions attached rather than a promise about generative AI in general.
Two documents help before the decision rather than after it. AWS AI Service Cards state the intended uses, limitations and responsible-design considerations for AWS AI services and models, in language a non-specialist can act on. Amazon SageMaker Model Cards do the equivalent job for a model the retailer trains or tunes itself. They record intended use, training data, evaluation results and known limitations in one place a reviewer can be pointed at.
Grounding the output in data the retailer owns
Amazon Bedrock Knowledge Bases indexes the product catalogue, the supplier spec sheets and the approved claims register. Every description is then generated from retrieved records rather than from the model’s general knowledge of pans. The technique is the one behind answering questions from your own documents, applied to writing rather than to answering. The citations-required arrangement matters more here than usual. A claim with no source record attached cannot be published.
Grounding does a second job that gets overlooked. A model rewriting a spec sheet the retailer owns has far less room to emit somebody else’s sentence than a model asked to invent copy from a product name, so the same control that reduces invented facts also narrows the surface for an infringement claim.
Filtering at the boundary
Amazon Bedrock Guardrails sits between the application and the model and applies the same policy whichever model is behind it. Content filters catch harmful categories at configurable strengths. Denied topics refuse subjects by description rather than by keyword, and word filters block specific competitor names or banned marketing phrases. Sensitive information filters redact anything that looks like personal data leaking in from a review.
The relevant piece for this catalogue is the contextual grounding check. It takes the source passages, the request and the model’s response, and returns two scores. A grounding score says how well the response is supported by the source. A relevance score says how well it answers what was asked. Set a threshold on each and the guardrail blocks anything that falls below. Applied to a generated description, that is an automatic refusal to publish a sentence the spec sheet does not support, which is the closest thing available to a machine check for hallucinations.
Marking what the model made
Amazon Nova image generation applies an invisible watermark to every image it produces, and Bedrock offers a detection call that reports whether an image carries one. Keeping that path open takes a deliberate decision. The watermark helps only if somebody in the organisation can still run the check in eighteen months, when a customer asks whether the campaign photograph was real.
The watermark answers one question. The retailer’s own record answers the rest, so log the model identity, the prompt, the retrieved sources, the timestamp and the person who approved it, alongside every published asset. That record is what turns “we think that one was generated” into an answer with a date on it. Amazon Rekognition content moderation gives a second pass over generated images before publication, catching the ones that came back with something nobody wants on a homewares page.
Human review, scoped narrowly
Nobody is going to staff a person to read 41,000 descriptions. Nobody has to. Route by claim type instead of by model confidence, because confidence is exactly the signal a hallucinating model gets wrong. Allergen and ingredient statements, safety and electrical ratings, age suitability, medical or health claims, warranty terms and price all go to a person before publication. Colour, dimensions, material and tone of voice publish automatically once they pass the grounding check. In this catalogue that split sends roughly 6,000 lines to review rather than 41,000, which is a fortnight of work for a small team instead of a hiring round.
Disclosure
Say that the content is AI-assisted, on the page, in the place a shopper is already looking, rather than in a policy document three clicks away. The disclosure line costs nothing and removes the discovery, and the discovery is where trust actually goes.
Evaluation
Side by side
| Risk | Model choice and licensing | Grounding in owned data | Guardrails | Watermark and record | Human review | Disclosure |
|---|---|---|---|---|---|---|
| Intellectual property infringement claims | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ |
| Biased model outputs | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ |
| Loss of customer trust | ✗ | ✓ | ✗ | ✓ | ✓ | ✓ |
| End user risk | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ |
| Hallucinations | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ |
Read the first column downwards and the argument the memo needs is already there. Model selection and its contract move exactly one row. That single row is worth moving, because an infringement claim is the one exposure here that arrives from outside with a lawyer attached. A team that spends three weeks comparing model licences and then ships without grounding has answered one fifth of the memo.
Read the human-review column and it ticks four rows. That is why the instinct to put a person on everything runs so strong, and why it has to be resisted on scope rather than on principle. A review step applied to 41,000 descriptions gets skimmed by week two and stops being a control. The same step applied to 6,000 regulated claims gets read.
The grounding column is the best value in the table. The guardrail column reaches its full strength only once the grounding column is filled, because a contextual grounding check with no retrieved source has nothing to score against.
What happens to one generated output
The ordering is doing the work. Traceability is checked before harm, because an untraceable claim is blocked whether or not it is dangerous. Asking a reviewer to adjudicate a sentence with no source record wastes the scarcest control in the system.
The solution
Start with the model and its paperwork, because that decision constrains everything downstream and is slow to reverse. Pick a hosted model from the Bedrock catalogue that carries intellectual property indemnity for generated output. Keep the guardrail filters on so the indemnity conditions hold. File the licence and the acceptable use policy alongside the design, so a reviewer can find them a year from now. If an open-weight model from SageMaker JumpStart is genuinely better for the catalogue, legal makes that decision with the licence in front of them rather than engineering making it on benchmark scores. The licensing terms belong in the comparison next to latency and cost. The same care applies going in. Forbid pasting competitor copy into a prompt, and do not fine-tune on anything whose provenance the team cannot state. That is the curated-sources question arriving in a legal register.
Then build the pipeline so the model never asserts an unsourced fact. Every description is generated from records retrieved out of a Bedrock Knowledge Base over the retailer’s own catalogue, spec sheets and approved-claims register. The prompt instructs the model to write only from the supplied records and to omit anything not present rather than fill the gap. A Bedrock guardrail wraps the call, with a contextual grounding check thresholded so an unsupported sentence fails, and word filters blocking competitor names and the marketing superlatives compliance already bans. Generated images pass through Rekognition moderation, and the invisible watermark plus your own generation log gives two independent answers to “where did this come from”.
Handle bias as a review of the corpus rather than of individual outputs, because it is invisible one description at a time and obvious across a category. Generate a sample, group it by product category, and read the tone. Then look at the same slices in the image set: who is in the kitchen, who is in the workshop, whose hands are on the appliance. Fixes come in two forms. Prompt-level constraints hold the register steady across categories, and a diversity brief on image prompts stops the generator defaulting to whatever its training corpus over-represents. Neither is a one-off. Re-run the category read after any model change, because a model swap resets every assumption about output tone.
Route the regulated claims to a person and let the rest publish. The classifier that decides which is which does not need to be clever. A claim register listing the attributes that require review, matched against the generated text, is more auditable than a model judging its own output. When a reviewer rejects something, capture why, because the rejection log is the evidence that the control operates and it is also the cheapest source of prompt improvements you will find.
Disclose. One line on the page saying descriptions are AI-assisted and reviewed, and a link to a short explanation of what that means. Then make it possible for a customer to reach a person about a specific page. That route back to a human turns a complaint into a correction rather than a story.
Three things bite afterwards. Indemnity conditions are conditions. Disabling a filter to clear a false positive on a Tuesday can quietly remove the protection the whole business case rested on, so filter changes go through review rather than a console toggle. Guardrail thresholds drift out of calibration as the catalogue changes, and a threshold set against homewares behaves differently over groceries. Sample the blocked outputs monthly and read what is being refused. And the review capacity has to be sized for the ongoing intake. Six thousand lines reviewed once is a project. Four hundred new lines a week arriving forever is a role somebody has to own.
Worked example
A supplier sheet for a cast-iron casserole lists the brand, a model number, 4.2 litres, enamelled cast iron, and dimensions. Nothing about hob compatibility, oven temperature or dishwasher suitability.
Ungrounded, the model returns a paragraph naming it induction-compatible, oven-safe to 260 degrees and dishwasher-safe. All three are plausible for enamelled cast iron and none of them appears on the sheet. The oven figure is the one that matters, because an enamel knob rated to 190 degrees fails at 260 and the shopper is holding a hot lid when it does. That is end user risk produced by hallucinations, and the paragraph reads beautifully.
Grounded against the catalogue record and the approved-claims register, the same request returns capacity, material, dimensions and colour, and omits the three claims the register does not carry. The contextual grounding check scores the response against the retrieved records and passes it. Hob compatibility is on the regulated list, so the field stays empty and a task goes to the merchandiser to ask the supplier. An empty field states what the retailer actually knows. The published page carries the AI-assisted line, and the generation log holds the model identity, the retrieved records and the timestamp.
Six weeks later a customer service enquiry asks whether the lifestyle photograph on that page is a real kitchen. The watermark detection call answers in a few seconds, and the generation log names the prompt and the person who approved it.
What’s worth remembering
- The legal risks of working with generative AI that this material names are intellectual property infringement claims, biased model outputs, loss of customer trust, end user risk, and hallucinations, and each one closes in a different place.
- Only intellectual property exposure moves meaningfully by choosing a model, through the provider’s indemnity and the terms accepted in the Bedrock model catalogue, on conditions that include leaving the filters switched on.
- Licensing is part of model selection, because an open-weight model from SageMaker JumpStart can carry prohibited uses, redistribution obligations and no indemnity at all, none of which changes because it runs on your own endpoint.
- Grounding generation in data you own, through Bedrock Knowledge Bases with a Guardrails contextual grounding check over it, reduces hallucinations, narrows end user risk and shrinks the surface for an infringement claim at once.
- Human review works when it is scoped to regulated claims such as allergens, safety ratings and health statements, and stops working when it is applied to every output.
- Watermarking generated images and logging the model, prompt and approver keeps the provenance question answerable later, and disclosing that content is AI-assisted removes the discovery that costs customer trust.