Checking a Bedrock Feature for Bias and Explainability

The situation

The applicant-summarisation feature has been in a private beta for a month. Given a candidate’s application text, it produces a three-sentence summary that a recruiter reads before deciding whether to advance the person, and a parallel path triages inbound support cases into priority tiers that route to human agents. Both outputs influence a decision about a person, so both land in scope for the responsible-AI review that gates launch.

The team has already wired up content safety. A guardrailGuardrailA filter or rule applied to an LLM’s inputs or outputs to keep it inside safe, legal, or on-brand behaviour. strips PII, blocks a list of denied topics, and runs a contextual grounding check so a summary can’t assert a qualification the application doesn’t support. That work was signed off weeks ago. The reviewers came back with two questions it doesn’t answer. Is the feature fair, meaning does it produce equal-quality summaries and equal treatment across groups of applicants, or does it write worse ones for some group without anybody noticing. And can a given output be explained, meaning if a recruiter or an auditor asks “why did it say that,” is there an answer.

Nobody on the team has measured either. They know how to enforce safety at runtime. Fairness and explainability are new jobs, and the first mistake would be to assume the guardrail already covers them.

What actually matters

AWS frames responsible AI as eight dimensions: fairness, explainability, privacy and security, safety, controllability, veracity and robustness, governance, and transparency. The guardrail the team already shipped covers safety, privacy, and part of veracity. Fairness and explainability are their own dimensions with their own controls, and for a generative feature they look nothing like the classic-classifier versions most people picture.

Bias in a generated summary isn’t the demographic parity of a single yes/no label. It shows up as stereotyping in the text, generalised statements that turn on a name, a school or a gender cue, and as disparate output quality across groups: richer, more favourable summaries for one cohort; thinner or more hedged ones for another; different refusal rates when the input carries a protected attribute. Measuring it means probing the model with inputs that vary the group signal and comparing what comes back, not counting positives and negatives in a confusion matrix.

Explainability for a foundation model is not SHAP-style per-feature attribution. You cannot hand a recruiter a bar chart of token weights and call it a justification. For a retrieval-augmentedRAGA pattern where you retrieve relevant documents at query time and stuff them into the prompt so the model can ground its answer on them. feature the practical form of explainability is traceability: which retrieved sources grounded the answer, surfaced as citations, so every claim in the summary points back to a line in the application. Alongside that sits documented behaviour (what the feature is for, where it fails) and a stated rationale. Promising feature attribution on an LLMLLMA neural network trained to predict the next token in a sequence, large enough that it generalises to tasks it wasn’t explicitly trained for. is a trap; traceability is the thing you can actually deliver.

The core failure is confusing four different jobs that all get filed under “responsible AI”:

• Content safety is a runtime job. Block, redact, and substitute the configured message as tokens flow.
• Bias measurement is an offline job. Score the model over a probe dataset before launch and again on a schedule.
• Explainability and traceability is a per-output job. Attach the sources and rationale to each answer as it’s produced.
• Human oversight is a high-stakes job. Route consequential outputs to a person before they act.

Reaching for a guardrail when the reviewer asked about fairness, or promising an explanation the model can’t produce, is how a review stalls. Match each question to the job that answers it.

What we’ll filter on

1. Fairness, does the control measure group fairness or stereotyping, not just single-label parity?
2. Explainability, does it produce a per-output explanation or source traceability?
3. Timing, is it a runtime enforcement control or an offline measurement one?
4. Safety, does it cover toxicity and unsafe content?
5. Transparency, does it emit a documentation artefact an auditor can read?
6. Oversight, does it support a human review step for high-stakes outputs?

The landscape

Amazon Bedrock Guardrails. Runtime content filters, denied topics, word and sensitive-information filters, a contextual grounding check that blocks ungrounded or irrelevant responses, and Automated Reasoning checks, which validate a response against a formal logic policy. This is enforcement for safety and controllability, applied as tokens flow. It reduces unsafe and off-topic output; it does not measure bias, and the grounding check addresses veracity (is the claim supported by the source), not fairness (is the treatment equal across groups). Configuring it is covered in setting up guardrails for PII, topics, and grounding.

Bedrock evaluations, programmatic. Offline scoring over a built-in or custom prompt dataset. The metrics here are accuracy, robustness and toxicity, nothing else; the built-in datasets include BOLD, which probes fairness in open-ended generation across profession, gender, race, religious ideology and political ideology, and RealToxicityPrompts. There is no stereotyping metric in a programmatic job.

Bedrock evaluations, judge model. A second model scores responses against built-in metrics, and stereotyping is one of them: Builtin.Stereotyping rates whether a response generalises about a group, positively or negatively, alongside Builtin.Harmfulness and Builtin.Refusal. A judge job can score inference responses you supply rather than invoking a model itself, so the summaries the feature actually produced get scored instead of fresh generations from a bare model. Both evaluation types are measurement, run before launch and on a schedule, not a runtime gate.

The fmeval library. SageMaker Clarify’s foundation-model evaluation is built on the open-source fmeval library, which scores factual knowledge, accuracy, toxicity, semantic robustness and prompt stereotyping over a dataset you supply, anywhere Python runs. Clarify itself closed to new customers on 30 July 2026; existing deployments keep running, and AWS points new builds at fmeval directly or at Bedrock evaluations. Clarify also computed classic-ML bias metrics and SHAP feature attribution, and that attribution is the classic-ML story; FM explainability is traceability and documentation, not feature attribution.

Grounding and citations. Bedrock Knowledge Bases RetrieveAndGenerate returns citations alongside the answer, so each generated claim traces to the source passage that supports it. For a RAG feature this is the working form of explainability: the recruiter sees which line of the application every sentence of the summary came from. If you override the default generation prompt template, keep the $output_format_instructions$ placeholder in it, because citations don’t appear in the response without it.

SageMaker Model Cards and AWS AI Service Cards. Transparency documentation. A Model Card records intended use, limitations, evaluation results, and known risks for your own model or feature. AI Service Cards are AWS’s own transparency documents for its managed AI services. The artefact is itself a control: an auditor reads it to understand what the feature is for and where it should not be trusted.

Human-in-the-loop review. A human review step routes high-stakes outputs to a person before they take effect. Amazon A2I (Augmented AI) used to package this loop; it closed to new customers on 30 July 2026, so existing loops keep running while a new one gets assembled from primitives (Step Functions or an SQS queue feeding your own reviewer UI) or uses a Bedrock human evaluation job with your own workforce. Either way, this is controllability for the consequential path: an applicant summary that will gate a rejection gets a human read, rather than acting unreviewed.

Watermarking and detection. For the multimodal case, images from Amazon Nova Canvas and Titan Image Generator carry an invisible watermark, and Bedrock’s watermark detection returns a confidence score for whether an image carries one. Detection is in public preview and limited to a few Regions, so treat it as provenance evidence rather than a gate. Not in scope for a text-summary feature, but it’s the transparency control when images enter the picture.

Evaluation

Side by side

No row does all of it. Fairness comes from the offline evaluators, per-output explainability from citations, safety from guardrails, transparency from the cards, oversight from human review, and evidence that any of it still holds next quarter from re-running the evaluators on a schedule and watching the scores in CloudWatch. The launch-ready answer is a stack, one control per job.

Dimensions to controls

The solution

The launch-ready answer is layered, and the layers run at different times.

Measure before launch. Run the evaluators over a probe dataset that varies the group signal. Stereotyping needs either a Bedrock judge-model job scoring Builtin.Stereotyping or the fmeval library’s prompt-stereotyping algorithm in your own pipeline; a programmatic Bedrock job covers toxicity and semantic robustness (does a trivial rewording of the input flip the summary) but has no stereotyping metric to offer. This is offline work that happens before a single real applicant is scored, and it repeats on a schedule because a model or prompt change can reintroduce bias.

Enforce at runtime. Keep the guardrail doing what it already does: strip PII, block denied topics, run the contextual grounding check so the summary can’t assert a qualification the application doesn’t support. Grounding and relevance each get a threshold between 0 and 0.99, and a response scoring under it is blocked rather than returned. The grounding check is a veracity control, not a fairness one; it stops fabrication, it doesn’t equalise treatment.

Explain per answer. Because the feature retrieves from the application text, wire RetrieveAndGenerate so every summary ships with citations. A recruiter, or an auditor six months later, can point at any sentence and see the source line. That is the explanation, and it’s one the feature can actually produce.

Document the behaviour. Fill in a SageMaker Model Card: intended use (recruiter decision support, not automated rejection), the factors affecting performance, evaluation results (the stereotyping and toxicity scores), ethical considerations, and a risk rating, which for a feature touching hiring decisions is high. Any edit other than an approval change versions the card, so the review reads an immutable record rather than a document someone tidied last night.

Oversee the high-stakes path. Route the consequential outputs, an applicant summary that feeds a reject decision, through a human review step before it acts: an SQS queue or a Step Functions task feeding a reviewer UI the team owns. Triage tiers that only change routing can run unattended; a summary that gates someone’s application should not.

The gotchas cluster around confusing the jobs. Do not promise SHAP-style feature attribution on the LLM; FM explainability is traceability and documentation. Bias in generation is stereotyping and quality parity, so the probe dataset has to actually vary the group signal, not just measure aggregate accuracy. Guardrails reduce unsafe output but never eliminate it, so measurement and human review still matter. The grounding check answers “is this claim supported,” which is not “is this treatment fair.” And the documentation is a control in its own right; the review is partly checking that it exists and is honest.

Keeping the number fresh

A stereotyping score measured the week before launch describes the model that shipped, not the one answering requests six months later. A reworded prompt, a new guardrail policy, a model-version bump: any of them can move the numbers, and none of that shows up unless something is measuring. So schedule the same fmeval or judge-model pass over a fixed probe set weekly, and fire it again on every prompt, guardrail or model-version change, so the change and its fairness result land together instead of a quarter apart.

Run a second pass over a sampled slice of the Bedrock model invocation logs as well. A judge job will score inference responses you hand it, so that sample goes in as-is. The fixed probe set is stable by design, which is what makes it comparable week to week, and stable also means it stops resembling the applications recruiters are actually submitting. Scoring real traffic keeps the measurement attached to the population the feature serves; the synthetic probes keep it comparable. Run both.

Publish what comes back: stereotyping score, refusal rate by cohort, favourability parity. Emit them as CloudWatch custom metrics so they sit on the same dashboard as latency and spend, and alarm on movement rather than on an absolute value. A stereotyping score of 0.09 says little on its own; 0.09 climbing to 0.21 across three weekly runs says a great deal. Movement is the signal, and watching for it is the continuous-governance half of the review that a launch-day report cannot cover.

SageMaker Model Monitor has a bias-drift feature, and it answers a different situation. It watches feature and label distributions crossing a SageMaker endpoint, comparing live traffic against a baseline computed from the training data. A Bedrock foundation model gives you none of those: no feature columns, no ground-truth labels arriving later, no training set you own. It is also closed to new customers as of 30 July 2026, so a team starting now cannot reach for it regardless. Here the scheduled evaluation job is the monitor.

Showing the reasoning to the user

Citations are raw material; how they get rendered decides whether anyone uses them. For a recruiter, the explanation is the summary arriving with the retrieved application passages beside it as source cards, each sentence linked to the line that supports it. They can open a card, read the applicant’s own words, and disagree with the summary on the spot. Source attribution in that form gets read by somebody with thirty seconds and a shortlist; a JSON citation block does not.

An execution trace serves a different reader. AgentCore Observability records spans for each step of an agent’s execution path, the tools called and the retrievals run, and stores them in CloudWatch, which is what an investigator opens weeks later when a candidate disputes a decision. Recruiters neither want it nor would work through it. Keep both, the source cards in the interface and the trace in CloudWatch, retained as long as the hiring records are. If the feature still runs on Bedrock Agents Classic, note that it closed to new customers on 30 July 2026 and AWS points new agent work at AgentCore.

Uncertainty is the third piece, and the guardrail already produces it. The contextual grounding check returns a grounding and a relevance confidence score per response, which is a signal you get without standing up a separate scorer. Collect them in CloudWatch and act on the low end. Left to the guardrail, a response under the threshold is blocked outright; call ApplyGuardrail yourself instead and the scores come back to you, which lets a weak answer go down the human review path rather than reaching the recruiter with a hedge stapled to it. A hedge reads as nuance, and nobody treats it as the warning it was meant to be.

Worked example

The team builds a probe dataset of 900 synthetic applications, matched in content but varying the group signal (name, pronoun, school prestige) across three cohorts of 300. They run an fmeval pass for prompt stereotyping and toxicity, and a robustness pass that rewords each input and checks whether the summary’s sentiment flips.

Prompt-stereotyping score (0 = no stereotyping, 1 = strong), by probe category:
  Gender-cued names        0.09
  School-prestige signal   0.07
  Pronoun swap             0.06

Toxicity rate (share of outputs flagged):
  Overall                  0.4%

Output-quality parity (mean summary "favourability" score, human-rated sample of 90):
  Cohort A                 4.11
  Cohort B                 4.08
  Cohort C                 3.74   ← gap

Refusal / hedge rate (response declines the task or hedges heavily):
  Cohort A                 2%
  Cohort B                 3%
  Cohort C                 9%     ← gap

Stereotyping and toxicity are low across the board, which is the easy pass. The problem surfaces in parity: cohort C, the one whose applications carried a lower-prestige school signal, gets thinner, more hedged summaries and a refusal rate three times the others. Aggregate accuracy hid it; the group-varied probe found it.

The mitigation follows from the gap. A promptPromptThe input you hand to an LLM – system instructions, user message, examples, retrieved documents, tool descriptions, the lot. change instructs the model to summarise qualifications on their own terms rather than relative to institution, and a re-run drops cohort C’s hedge rate to 4% and closes the favourability gap to 0.1. Because the gap touches a launch-gating decision, the applicant-summary path also goes behind human review, and a new version of the Model Card records the cohort-C finding and the mitigation so the next reviewer sees the history. The triage-tier path, which only changes routing, launches without the review step. The feature ships, and it ships with the evidence that it was checked.

What’s worth remembering

1. Content safety, bias measurement, explainability, and human oversight are four distinct jobs at different times; confusing them is the core failure.
2. Bias in a generative feature is stereotyping and output-quality parity across groups, not the demographic parity of a single label.
3. Measure fairness offline before launch and on a schedule, with a Bedrock judge-model job for Builtin.Stereotyping or the fmeval library for prompt stereotyping; a programmatic Bedrock job gives you accuracy, robustness and toxicity only.
4. FM explainability is traceability and documentation, not feature attribution; do not promise SHAP on an LLM.
5. For a RAG feature, per-answer citations from RetrieveAndGenerate are the working form of explainability.
6. Probe with a dataset that varies the group signal, because aggregate accuracy hides the gap that a group-varied run exposes.