Instrumenting a Bedrock application tells you what happened to the requests that arrived. Synthetics covers the gap that leaves: an endpoint nobody called overnight looks identical to a healthy one until the first subscriber of the morning finds it broken. A canary manufactures the traffic, so the alarm fires at 3am on a schedule you chose rather than at 9am on a complaint.
Flash card
Amazon CloudWatch Synthetics: a scheduled Node.js, Python or Java canary that exercises a live endpoint from outside and reports pass or fail, availability and latency as CloudWatch metrics.
- A canary is a script on a schedule, as often as once a minute, hitting the real endpoint through the real auth, network and dependencies, so the feature produces operational metrics even when no subscriber touches it.
- Each run publishes SuccessPercent and Duration to the CloudWatchSynthetics namespace to alarm on, sends its logs to CloudWatch Logs, and stores screenshots and an HTTP archive in S3 for every run, which is what turns a red alarm into a diagnosis.
- For a generative feature the canary sends one fixed golden prompt and asserts on the shape of the answer, the presence of a citation, or a similarity band against a recorded reference, because exact text will not hold; that catches a broken deploy, an expired credential, a regional model outage, or a latency regression before a user finds it.
- A canary can run in a VPC to reach a private endpoint, given either internet access or VPC endpoints for CloudWatch and S3, and runtimes from syn-nodejs-2.0 onwards support active X-Ray tracing, so a failed run has a trace behind it.
- Each run is billed, and a canary against a generative endpoint spends Bedrock tokens as well, so the interval you set fixes both the bill and the delay before an outage shows up, and the same prompt on a cadence gives you a latency baseline.
Pick it when
Pick it for outside-in availability and latency on a low-traffic feature, for a deployment validation smoke test straight after a prompt or model version change, and wherever a synthetic workflow has to keep exercising a journey while traffic is quiet.
It's the wrong answer when
It is the wrong choice for measuring what real users experienced, which needs client-side instrumentation of real sessions; for scoring answer quality across a dataset, which is an evaluation job over a golden set and invocation logs; and for tracing one failed production request, which is X-Ray. A canary is a heartbeat; more than a handful of fixed cases belongs in an evaluation harness instead.