The situation
A public-facing assistant runs an Amazon Bedrock model behind Amazon API Gateway, with Amazon Cognito issuing the tokens that identify each user. Signing up is free, which is a product decision and also the reason this scenario exists. Anyone with an email address can get an identity and start asking questions.
Three weeks after launch, the pattern in the logs is ugly in three separate ways. One identity trips the guardrail’s prompt-attack filter about forty times an hour, every hour, in what looks like a scripted sweep. A second runs roughly ten times the median tokens per session, pasting enormous documents in and asking for full rewrites. A third asks the same off-policy question in twenty different phrasings until one of them gets past the denied-topic filter, then goes quiet for a day and starts again.
No control found any of this. An engineer scrolling through model invocation logs on a Friday afternoon did. The team has dashboards and traces for the healthy path and guardrails in the invocation path, so the raw material exists. What is missing is anything that watches the material without a human in front of it. Nobody has agreed what the system may do by itself when it finds something.
What actually matters
Detection and response are two decisions, and running them together is how a team ends up with a dashboard nobody looks at. A detection decision asks what evidence exists that a pattern is abuse. A response decision asks what the system may do about it without asking permission. The second one is where the argument actually is. Almost any team will agree to more logging. Far fewer will agree, in advance, that a rule may cut off a paying customer at three in the morning.
The signals are not interchangeable, because they observe different things. A volume signal counts requests, bytes and tokens; it returns an answer in milliseconds and carries nothing about the content. A semantic signal reports that the content was a prompt attack or an off-policy topic, because a filter evaluated it, and producing it takes an extra evaluation in the request path. A sequence signal is the hardest of the three. Twenty rephrasings of the same off-policy question only appear when a whole session is examined together, and neither a per-request volume counter nor a per-request content filter covers a session. The third user in this scenario is invisible to the two fastest controls, which is roughly why they chose that approach.
Attribution decides how useful any of this is. A signal that cannot be tied to an identity can only produce a global response, and a global response reaches everyone in order to reach one person. Rate limiting by source address throttles a shared corporate NAT gateway along with the one user behind it. Every signal here is limited by whether it can name a Cognito subject, and getting that name into the signal is something the application has to do deliberately.
Then there is the asymmetry in being wrong. A false positive that raises an alert takes an engineer five minutes. A false positive that throttles an identity gives a customer a slow afternoon. A false positive that disables a Cognito user removes the product from that customer at the moment they were using it hardest. The people most likely to look statistically abnormal are power users doing exactly what the thing was built for. So the response gets graduated by how confident the signal is. Every automated action leaves an audit record naming the rule that fired and the evidence behind it, and a suspension needs a route back for someone who did nothing wrong.
What we’ll filter on
- What the signal covers: request volume, prompt or response semantics, or a pattern across a session.
- Attribution: does the signal arrive carrying an end-user identity, does the application have to stamp one on, or can it never carry one?
- Timing: does it act before the response returns, within a minute or two, or only when somebody runs a query afterwards?
- Baseline: does it need a fixed threshold somebody has to guess, or a learned band that needs history behind it?
- Response tier the signal can safely trigger on its own, given how often it will be wrong.
The landscape
Guardrail metrics
Amazon Bedrock Guardrails publish metrics to CloudWatch in the AWS/Bedrock/Guardrails namespace, among them InvocationsIntervened and TextUnitCount. Both carry a GuardrailPolicyType dimension whose values are ContentPolicy, TopicPolicy, WordPolicy, SensitiveInformationPolicy and ContextualGroundingPolicy, and a GuardrailContentSource dimension separating input from output. A rise in denied-topic interventions is therefore distinguishable from a rise in contextual-grounding failures, and an input-side spike from an output-side one.
Two limits matter. The prompt-attack filter is one of the content filters, so its interventions roll up under ContentPolicy alongside hate, insults, sexual, violence and misconduct; the metric will not separate a scripted jailbreak sweep from a surge of abusive language. And no dimension names a user. The available dimensions are the operation, the content source, the policy type, and the guardrail ARN and version. A guardrail metric reports that the application intervened forty times, not that one subject caused all forty.
Model invocation logs
Bedrock model invocation logging records the request and response, delivered to CloudWatch Logs, to Amazon S3, or to both. Every field is populated by Bedrock automatically except one. requestMetadata is a caller-supplied object of up to sixteen key-value pairs, set as the requestMetadata field on Converse or the X-Amzn-Bedrock-Request-Metadata header on InvokeModel. That is where the Cognito subject goes. The record also carries identity.arn, but for a public assistant every call runs under the same application role, so the ARN names the application and only the metadata names the user.
Sent to CloudWatch Logs, the records are queryable in Logs Insights, which suits an on-call engineer chasing a pattern from the last few hours. Sent to S3, they arrive as gzipped JSON that Amazon Athena can query over months. Request and response bodies up to 100 KB are inline. Anything larger, including the hundred-page contract the second user is pasting, is written as a separate S3 object, with a reference in the record. Response logging matters as much as prompt logging, because a jailbreak is only confirmed by what came back.
Anomaly detection on the metrics
CloudWatch anomaly detection applies a model to a metric’s own history and draws a band of expected values around it, accounting for hourly, daily and weekly seasonality as well as trend. That catches token burst patterns without anybody guessing a number. The algorithm trains on up to two weeks of data, and you can enable it on a metric with less than that behind it, so the band tightens as history accumulates rather than arriving finished. You can also exclude specified time periods from training, which is how a launch or a marketing push stops distorting the baseline.
AWS WAF in front of the API
AWS WAF sits on the API Gateway stage. Its rate-based rules count requests over an evaluation window and rate limit above a limit, and its managed rule groups cover the general web nastiness unrelated to this application. It evaluates the request before it reaches your code, which makes it the fastest available stop for volume abuse, and it does not inspect prompt semantics in any useful way.
The default aggregation is the source IP address, which is blunt against anyone behind a shared egress. That is not the only option. A rate-based rule can aggregate on custom keys, among them a named header, a named cookie, a query argument, a label namespace or a JA4 fingerprint, and it can combine several. If the application puts a stable per-subscriber value in a header, WAF can rate limit that subscriber rather than their network.
API Gateway usage plans and per-key throttling
A usage plan attaches a request-rate limit, a burst limit and a quota over a day, a week or a month to an API key, and those limits apply per key across the stages in the plan. The throttle lands on an identity rather than a network location, and it is the control that a gateway in front of Bedrock usually already has wired.
Two caveats, both from AWS. Usage plan throttling and quotas are best-effort rather than hard limits, clients can exceed them, and the documentation advises against relying on them to block access or control cost. And an API key is not authentication: it identifies a client for metering, while Cognito and an authorizer do the access control.
CloudTrail
CloudTrail logs InvokeModel, InvokeModelWithResponseStream, Converse and ConverseStream as management events, which are recorded by default rather than switched on. The record names the principal, the model, the source address and the time. For a public assistant most invocations run under one application role, so it will not distinguish a chatty end user, but it will surface a role nobody expected calling models, which is the access-governance question rather than the abuse one.
Some Bedrock operations are data events instead, and those have to be enabled with advanced event selectors. ApplyGuardrail is one, on the AWS::Bedrock::Guardrail resource type, and it covers guardrail evaluations made during model invocation. The event body carries the assessment: which filter type matched, at what confidence, and whether the action was BLOCKED. That is semantics with a principal attached, though still the application’s principal. One caveat from the documentation: when more than one guardrail evaluates an invocation, the event does not identify which guardrail produced which assessment, so don’t attribute by position.
The response side
The second axis has four rungs, ordered by how much damage each does when it is wrong. Alert only, which notifies a human and changes nothing. Throttle, which restricts the offending identity for a set period and lapses on a timer. Suspend, which disables the Cognito user so their tokens stop working. And route to human review, which parks the evidence in a queue for a person to decide, the right destination for anything expensive and ambiguous. Amazon Augmented AI was the managed option for that step; it closed to new customers on 30 July 2026, so a new build wires its own queue.
Wiring is where these stop being policy statements. CloudWatch sends an event to Amazon EventBridge whenever an alarm changes state, with guaranteed delivery and a CloudWatch Alarm State Change detail type, and EventBridge routes to an AWS Step Functions state machine. The state machine calls Lambda functions to do the work: read the recent invocation logs for that subject, pick the tier, apply the restriction, write the audit record, and notify both the user and the on-call channel. A workflow built this way leaves an execution history for every decision, which is what makes the remediation auditable.
The pre- and post-processing filters run alongside all of it. Amazon Comprehend can screen an input before it reaches the model: DetectToxicContent scores categories including harassment or abuse, hate speech and violence or threat, in English only, and DetectPiiEntities handles English and Spanish. Guardrails apply the model-based checks in the invocation path. On the way back, a Lambda function in the integration validates the completion before API Gateway returns it, catching anything the earlier layers passed, and the exfiltration controls live in the same place.
Evaluation
Side by side
| Signal | Carries content | Names the end user | Acts before the response returns | Needs a baseline period | Highest tier it should trigger alone |
|---|---|---|---|---|---|
| WAF rate-based rules and managed rule groups | ✗ | ✓ with a custom aggregation key | ✓ | ✗ | Throttle or block |
| API Gateway usage plans, per-key throttling | ✗ | ✓ (per key, best-effort) | ✓ | ✗ | Throttle |
| Guardrail CloudWatch metrics by policy type | ✓ | ✗ (no user dimension) | ✓ | ✗ | Alert |
| CloudWatch anomaly detection bands | ✗ | ✗ | ✗ | ✓ | Alert |
| Invocation logs in Logs Insights and Athena | ✓ | ✓ via requestMetadata |
✗ | ✗ | Human review |
| CloudTrail management and guardrail data events | ✓ (assessments) | ✗ (application role) | ✗ | ✗ | Alert |
Read the last two columns together. The controls that evaluate fastest carry the least about content, and the record that carries the most is queried after the fact. No row does both jobs. The tier column is the operational consequence: a signal that cannot name a subject has no business suspending one.
Signal to control to response tier
The solution
Layer the detection so each of the three patterns has a control that covers it, and graduate the response so the severity matches how confident the signal is.
Detection, layered
WAF rate-based rules go on the API Gateway stage as the outer layer, with the managed rule groups switched on for generic web traffic. Aggregate on a custom key, a header carrying a stable per-subscriber value, rather than leaving the default source IP, so a shared corporate egress is not rate limited as one client. API Gateway usage plans sit behind that with a per-key rate, burst and daily quota, treated as metering and shaping rather than as a hard stop, because AWS documents them as best-effort.
Guardrail metrics carry the semantic layer at the aggregate level, split by policy type and content source, and they are the fastest warning that something has changed. They cannot name the subject, so the per-subject semantic evidence comes from the invocation logs. Make the application set requestMetadata with the Cognito subject on every Bedrock call, and turn on invocation logging to both CloudWatch Logs and S3: Logs Insights for the last few hours, Athena over the S3 copy for the session-shaped queries and the retention window compliance wants. A CloudWatch alarm can run a scheduled Logs Insights query with a grouping clause and alarm per contributor. That turns a per-subject intervention count into an alarm within minutes, rather than a query someone has to remember to run.
Anomaly detection bands go on tokens per invocation and invocations per minute. Token burst patterns appear there without anybody guessing a threshold that will be wrong by next month. Enable them early and let the model accumulate its two weeks rather than waiting, and exclude launch windows from training instead of letting them widen the band. CloudTrail management events already cover the caller side; add guardrail data events when you want the per-filter assessments in the same trail.
The scheduled Athena query is the piece people leave out, and it is the only thing in the design that catches the third user.
Response, graduated
Every detection publishes to EventBridge. A Step Functions state machine reads the event, gathers context from the invocation logs for that subject, and picks a tier.
Tier one is alert only, and it is where anything unattributed lands. An anomaly band firing on aggregate tokens gets a notification and a dashboard link, because there is no subject to act against.
Tier two is a throttle. Move that subscriber’s API key to a restricted usage plan rather than editing the shared plan, which would land on every key attached to it. Add their aggregation key to a WAF rule when the traffic has to stop rather than slow. The restriction lapses on expiry rather than on someone remembering, and the user gets told what happened and why.
Tier three is human review. The state machine writes the evidence, the matching prompts and responses, and the rule that fired into a review queue, and stops. Anything that would end an account goes through here, because being wrong here ends the relationship.
Tier four is suspension of the Cognito user. It fires automatically only for the narrow, high-confidence cases agreed in advance, such as a subject over a hard interventions-per-hour threshold with a matching prompt-attack signature. It always writes an audit record, always notifies the user, and always creates a review case so a human confirms it afterwards.
The gotchas
Guardrail metrics have no user dimension, and no request attribute adds one. Request metadata reaches the invocation logs and never the metric. This is the most common gap: the team builds a per-policy dashboard, watches the interventions climb, and cannot answer which of forty thousand accounts is responsible. The answer is a query or a log alarm over the invocation logs, not a better dashboard.
That same metric will not separate a prompt-attack sweep from a rise in abusive language, because prompt attacks are one of the content filters and all of them report under ContentPolicy. Splitting them needs the per-request assessment, from the guardrail trace, the invocation log, or an ApplyGuardrail data event.
WAF does not inspect prompt semantics, so it counts volume and nothing else. A patient attacker sending one carefully crafted prompt every ten minutes stays under it entirely, and the adversarial testing exercise that found the jailbreak will tell you exactly how patient they need to be.
Anomaly bands need history. Alarming on a band with three days behind it produces noise and teaches the on-call to ignore it.
Automatic suspension needs an appeal path. The legitimate power user who pastes a hundred-page contract in every morning looks like the token-burn attacker until a human reads the prompts. Without a way back, the automation turns an unusual customer into a churned one overnight.
And the response workflow has to be logged as carefully as the assistant is. A Lambda function that disables accounts and writes nothing down is a worse governance problem than the misuse it was built to stop. Tagging the detection and response stack alongside the rest of the workload keeps its own cost visible too, which matters once Athena is scanning months of logs on a schedule.
Worked example
Take the three offenders in order.
The scripted sweep tripping the prompt-attack filter forty times an hour shows up first as a rise in InvocationsIntervened for ContentPolicy on the input side. That is a warning and not an accusation. The scheduled Logs Insights query over the invocation log group counts interventions by requestMetadata subject, and alarms on the one contributor above the threshold. EventBridge carries the alarm state change, Step Functions confirms the pattern against the last hour of logs, and it applies a tier-two throttle plus a tier-three review case. The user gets a message saying their requests are being rate limited; a human confirms within the day and moves it to suspension.
The token burner shows up on the anomaly band for tokens per invocation, which alarms at tier one because the band is an aggregate. That alert is enough for the state machine to run a targeted Athena query grouping the last day by subject, which names them immediately. The evidence goes to review rather than to an automatic action, and the reviewer finds a translation agency running exactly the workload the product is for. The outcome is a sales conversation and a higher usage-plan tier, not a suspension. That is the design working.
The patient rephraser trips nothing in real time, because each individual prompt is unremarkable and the volume is low. The nightly Athena query over the S3 invocation logs groups by subject and by embedding neighbourhood of the prompt. It finds twenty near-identical asks against one denied topic, and the one response that got through. That last part rests on response logging. The query proves a policy violation happened, not only that one was attempted. The case goes straight to human review, and the surviving completion goes to whoever owns the denied-topic wording, because the fix is a better guardrail, not a banned account.
What’s worth remembering
- Detection and response are two separate decisions, and continuous monitoring is only useful when the second one has been agreed in advance rather than improvised during an incident.
- Guardrail CloudWatch metrics break down by policy type and content source but have no user dimension. Request metadata reaches the model invocation logs rather than the metric, so per-subject semantic evidence comes from a query or a log alarm over those logs.
- Prompt-attack interventions report under
ContentPolicyalongside the other content filters, so separating a jailbreak sweep from abusive language needs the per-request assessment. - CloudTrail records Bedrock invocations as management events by default,
ApplyGuardrailis a data event you switch on, and both name the application’s principal rather than the end user. - CloudWatch anomaly detection trains on up to two weeks of data and lets you exclude launch windows from training, which is what keeps a token-burst band usable.
- API Gateway usage plan throttles and quotas are best-effort by AWS’s own documentation, so a hard stop belongs in WAF. Route every automated action through EventBridge to Step Functions so it has an execution history and a reversal path.