Checking a Bedrock Feature for Bias and Explainability

The situation

The applicant-summarisation feature has been in a private beta for a month. Given a candidate’s application text, it produces a three-sentence summary that a recruiter reads before deciding whether to advance the person, and a parallel path triages inbound support cases into priority tiers that route to human agents. Both outputs influence a decision about a person, so both land in scope for the responsible-AI review that gates launch.

The team has already wired up content safety. A guardrailGuardrailA filter or rule applied to an LLM’s inputs or outputs to keep it inside safe, legal, or on-brand behaviour. strips PII, blocks a list of denied topics, and runs a grounding check so the model doesn’t invent qualifications the applicant never claimed. That work was signed off weeks ago. The reviewers came back with two questions it doesn’t answer. Is the feature fair, meaning does it produce equal-quality summaries and equal treatment across groups of applicants, or does it quietly write worse summaries for some. And can a given output be explained, meaning if a recruiter or an auditor asks “why did it say that,” is there an answer.

Nobody on the team has measured either. They know how to enforce safety at runtime. Fairness and explainability are new jobs, and the first mistake would be to assume the guardrail already covers them.

What actually matters

AWS frames responsible AI as eight dimensions: fairness, explainability, privacy and security, safety, controllability, veracity and robustness, governance, and transparency. Content safety touches maybe three of them. Fairness and explainability are their own dimensions with their own controls, and for a generative feature they look nothing like the classic-classifier versions most people picture.

Bias in a generated summary isn’t the demographic parity of a single yes/no label. It shows up as stereotyping in the text (the model leaning on assumptions tied to a name, a school, a gender cue) and as disparate output quality across groups (richer, more favourable summaries for one cohort; thinner or more hedged ones for another; different refusal rates when the input mentions a protected attribute). Measuring it means probing the model with inputs that vary the group signal and comparing what comes back, not counting positives and negatives in a confusion matrix.

Explainability for a foundation model is not SHAP-style per-feature attribution. You cannot hand a recruiter a bar chart of token weights and call it a justification. For a retrieval-augmentedRAGA pattern where you retrieve relevant documents at query time and stuff them into the prompt so the model can ground its answer on them. feature the practical form of explainability is traceability: which retrieved sources grounded the answer, surfaced as citations, so every claim in the summary points back to a line in the application. Alongside that sits documented behaviour (what the feature is for, where it fails) and a stated rationale. Promising feature attribution on an LLMLLMA neural network trained to predict the next token in a sequence, large enough that it generalises to tasks it wasn’t explicitly trained for. is a trap; traceability is the thing you can actually deliver.

The core failure is confusing four different jobs that all get filed under “responsible AI”:

• Content safety is a runtime job. Block, redact, and refuse as tokens flow.
• Bias measurement is an offline job. Score the model over a probe dataset before launch and again on a schedule.
• Explainability and traceability is a per-output job. Attach the sources and rationale to each answer as it’s produced.
• Human oversight is a high-stakes job. Route consequential outputs to a person before they act.

Reaching for a guardrail when the reviewer asked about fairness, or promising an explanation the model can’t produce, is how a review stalls. Match each question to the job that answers it.

What we’ll filter on

1. Fairness, does the control measure group fairness or stereotyping, not just single-label parity?
2. Explainability, does it produce a per-output explanation or source traceability?
3. Timing, is it a runtime enforcement control or an offline measurement one?
4. Safety, does it cover toxicity and unsafe content?
5. Transparency, does it emit a documentation artefact an auditor can read?
6. Oversight, does it support a human review step for high-stakes outputs?

The responsible-AI landscape

1. Amazon Bedrock Guardrails. Runtime content filters, denied topics, word and PII filters, a contextual grounding check that flags ungrounded or hallucinated output, and automated reasoning checks that test output against a formal policy. This is enforcement for safety and controllability, applied as tokens flow. It reduces unsafe and off-topic output; it does not measure bias, and the grounding check addresses veracity (is the claim supported by the source), not fairness (is the treatment equal across groups). Configuring it is covered in setting up guardrails for PII, topics, and grounding.

2. Bedrock model evaluation, automated. Offline scoring over a dataset. Built-in metrics include toxicity and, for bias, a prompt-stereotyping metric that measures whether the model prefers stereotype-consistent continuations. Also scores robustness. This is measurement, run before launch and on a schedule, not a runtime gate.

3. Amazon SageMaker Clarify. For foundation models, Clarify’s FM evaluation (the fmeval library) scores accuracy, toxicity, semantic robustness, and prompt stereotyping for bias, over a dataset you supply. For classic ML models Clarify also computes pre- and post-training bias metrics and SHAP feature attribution, but that attribution is the classic-ML story; FM explainability is traceability and documentation, not feature attribution. Use Clarify’s FM path for a generative feature.

4. Grounding and citations. Bedrock Knowledge Bases RetrieveAndGenerate returns citations alongside the answer, so each generated claim traces to the source passage that supports it. For a RAG feature this is the working form of explainability: the recruiter sees which line of the application every sentence of the summary came from.

5. SageMaker Model Cards and AWS AI Service Cards. Transparency documentation. A Model Card records intended use, limitations, evaluation results, and known risks for your own model or feature. AI Service Cards are AWS’s own transparency documents for its managed AI services. The artefact is itself a control: an auditor reads it to understand what the feature is for and where it should not be trusted.

6. Human-in-the-loop review. Amazon A2I (Augmented AI) or a hand-built human review step routes high-stakes outputs to a person before they take effect. This is controllability for the consequential path: an applicant summary that will gate a rejection gets a human read, rather than acting unreviewed.

7. Amazon Titan image watermarking and detection. For the multimodal case, generated images carry an invisible watermark with a detection API, giving provenance and transparency for synthetic media. Not in scope for a text-summary feature, but it’s the transparency control when images enter the picture, as covered in making a chatbot audit-ready with guardrails and watermarks.

Side by side

No row does all of it. Fairness comes from the offline evaluators, per-output explainability from citations, safety from guardrails, transparency from the cards, oversight from human review. The launch-ready answer is a stack, one control per job.

Dimensions to controls

The pick in depth

The launch-ready answer is layered, and the layers run at different times.

Measure before launch. Run Clarify’s FM evaluation, or the equivalent Bedrock model-evaluation job, over a probe dataset that varies the group signal. Score prompt stereotyping (does the model prefer stereotype-consistent output), toxicity, and semantic robustness (does a trivial rewording of the input flip the summary). This is offline work that happens before a single real applicant is scored, and it repeats on a schedule because a model or prompt change can reintroduce bias.

Enforce at runtime. Keep the guardrail doing what it already does: strip PII, block denied topics, run the contextual grounding check so the summary can’t assert a qualification the application doesn’t support. The grounding check is a veracity control, not a fairness one; it stops fabrication, it doesn’t equalise treatment.

Explain per answer. Because the feature retrieves from the application text, wire RetrieveAndGenerate so every summary ships with citations. A recruiter, or an auditor six months later, can point at any sentence and see the source line. That is the explanation, and it’s one the model can actually stand behind.

Document the behaviour. Fill in a Model Card: intended use (recruiter decision support, not automated rejection), limitations (the groups where evaluation showed weaker performance), evaluation results (the stereotyping and toxicity scores), and known risks. The card is a control the review can read, not overhead bolted on afterwards.

Oversee the high-stakes path. Route the consequential outputs, an applicant summary that feeds a reject decision, through a human review step with A2I before it acts. Triage tiers that only change routing can run unattended; a summary that gates someone’s application should not.

The gotchas cluster around confusing the jobs. Do not promise SHAP-style feature attribution on the LLM; FM explainability is traceability and documentation. Bias in generation is stereotyping and quality parity, so the probe dataset has to actually vary the group signal, not just measure aggregate accuracy. Guardrails reduce unsafe output but never eliminate it, so measurement and human review still matter. The grounding check answers “is this claim supported,” which is not “is this treatment fair.” And the documentation is a control in its own right; the review is partly checking that it exists and is honest.

A worked example: the probe run before launch

The team builds a probe dataset of 900 synthetic applications, matched in content but varying the group signal (name, pronoun, school prestige) across three cohorts of 300. They run a Clarify FM evaluation for prompt stereotyping and toxicity, and a robustness pass that rewords each input and checks whether the summary’s sentiment flips.

Prompt-stereotyping score (0 = no stereotyping, 1 = strong), by probe category:
  Gender-cued names        0.09
  School-prestige signal   0.07
  Pronoun swap             0.06

Toxicity rate (share of outputs flagged):
  Overall                  0.4%

Output-quality parity (mean summary "favourability" score, human-rated sample of 90):
  Cohort A                 4.11
  Cohort B                 4.08
  Cohort C                 3.74   ← gap

Refusal / hedge rate (model declines or heavily hedges):
  Cohort A                 2%
  Cohort B                 3%
  Cohort C                 9%     ← gap

Stereotyping and toxicity are low across the board, which is the easy pass. The problem surfaces in parity: cohort C, the one whose applications carried a lower-prestige school signal, gets thinner, more hedged summaries and a refusal rate three times the others. Aggregate accuracy hid it; the group-varied probe found it.

The mitigation follows from the gap. A promptPromptThe input you hand to an LLM – system instructions, user message, examples, retrieved documents, tool descriptions, the lot. change instructs the model to summarise qualifications on their own terms rather than relative to institution, and a re-run drops cohort C’s hedge rate to 4% and closes the favourability gap to 0.1. Because the gap touches a launch-gating decision, the applicant-summary path also goes behind human review, and the Model Card records the cohort-C finding and the mitigation so the next reviewer sees the history. The triage-tier path, which only changes routing, launches without the review step. The feature ships, and it ships with the evidence that it was checked.

What’s worth remembering

1. The eight responsible-AI dimensions map to concrete, different AWS controls; there is no single “responsible AI” switch, and fairness and explainability are not the guardrail you already built.
2. Content safety, bias measurement, explainability, and human oversight are four distinct jobs at different times; confusing them is the core failure.
3. Bias in a generative feature is stereotyping and output-quality parity across groups, not the demographic parity of a single label.
4. Measure fairness offline with Clarify FM evaluation or Bedrock model evaluation (prompt stereotyping, toxicity, robustness), before launch and on a schedule.
5. FM explainability is traceability and documentation, not feature attribution; do not promise SHAP on an LLM.
6. For a RAG feature, per-answer citations from RetrieveAndGenerate are the working form of explainability.
7. The contextual grounding check is a veracity control (is the claim supported), not a fairness one (is the treatment equal).
8. A Model Card is a control, not paperwork: intended use, limitations, evaluation results, and known risks, written down for the reviewer and the auditor.
9. Route high-stakes outputs through human review with A2I; let low-stakes routing run unattended.
10. Probe with a dataset that varies the group signal, because aggregate accuracy hides the gap that a group-varied run exposes.