Exam-style
Before a generative feature launches, the legal team wants a measured report on toxicity and bias across a 2,000-prompt test set, using a foundation model the company did not train. What produces that report?
Reveal the answer
A. SageMaker Clarify foundation-model evaluation, scoring toxicity, prompt stereotyping, accuracy, and semantic robustness over the prompt set
Clarify’s foundation-model evaluation, the fmeval library, is built to score generated text on toxicity, prompt stereotyping, accuracy, and robustness, which is the shape bias takes for a generative model; Bedrock model evaluation covers similar ground. Guardrails enforce at runtime and produce blocks rather than measurements. The classical pre-training bias metrics need labelled training data the team does not have for someone else’s foundation model, and Comprehend sentiment measures tone rather than stereotyping or quality disparity.
Q. You must measure a GenAI feature for bias and toxicity before launch. Which service?
A. SageMaker Clarify’s foundation-model evaluation (the fmeval library) scores accuracy, toxicity, semantic robustness, and prompt stereotyping (bias). Bedrock model evaluation also offers toxicity and stereotyping metrics.
Why? Bias for a generative model shows up as stereotyping and quality disparity, measured offline, not as label parity.