Exam Room · AI Practitioner

Pop Quiz: Which Log Records What Was Said

· 6 min read

Exam-style

A customer complains that a Bedrock-backed assistant promised them a refund the company does not offer, and quotes the reply. The team believes the assistant never produced that text. AWS CloudTrail is on, Amazon CloudWatch metrics and alarms are in place, and the application writes its own logs. Which record reconstructs the actual exchange?

Reveal the answer

B. Amazon Bedrock model invocation logging, which captures the prompt sent and the completion returned and delivers them to a destination the team owns

Only Amazon Bedrock model invocation logging stores the text. It captures the request and response bodies, prompt and completion, and delivers them to an Amazon S3 bucket or an Amazon CloudWatch Logs group in the same account and Region. It is disabled by default, so teams find they need it on the day somebody asks what was said. AWS CloudTrail records the API call rather than its content: which principal called InvokeModel, from which account and Region, at what time, against which model ID. It never carries a sentence the model returned. Amazon CloudWatch carries counts: Invocations, InvocationLatency, error and throttle counts, and the guardrail metric InvocationsIntervened. Those drive alarms and dashboards, and none is a transcript. The application logs hold the request that reached the service: what the customer typed plus whatever context the code attached; the completion comes back from Bedrock and appears there only if somebody wrote code to log it, scattering generated text into stores nobody secured for it. Replaying the question returns a fresh answer rather than a copy of the first one, and the retrieved context may have moved on since. Once invocation logging is on, that destination holds every prompt and every answer, so it needs its own encryption key, access policy and retention setting.

AI Fundamentals · part of The Exam Room

Q. A customer quotes a reply the assistant supposedly gave. CloudTrail, CloudWatch and application logs are all running. Which record holds the exchange?

A. Amazon Bedrock model invocation logging, which captures the prompt and the completion and writes them to a bucket or log group you own. It is off until somebody switches it on.

Why? Sort the four logs by what each one holds. AWS CloudTrail holds the call: principal, time, Region, model ID, no text. Amazon CloudWatch holds the numbers: invocations, latency, errors, guardrail interventions, the material for alarms and dashboards. Application logs hold the request that arrived at your service, which is application security territory rather than the model’s answer. Invocation logging holds the words, and nothing else does. Audit trail and logging requirements for AI interactions come down to two records: one for who called, one for what was said. Turning the second one on moves the sensitive data problem rather than solving it, so give that destination a customer-managed key, a tight access policy and a retention rule before the logs start arriving.

These posts are LLM-aided. Backbone, original writing, and structure by Craig. Research and editing by Craig + LLM. Proof-reading by Craig.