Instrumenting a Bedrock application tells you what happened to the requests that arrived. Synthetics covers the gap that leaves: an endpoint nobody called overnight looks identical to a healthy one until the first subscriber of the morning finds it broken. A canary manufactures the traffic, so the alarm fires at 3am on a schedule you chose rather than at 9am on a complaint.
Flash card
Amazon CloudWatch Synthetics: a scheduled Node.js, Python or Java canary that exercises a live endpoint from outside and reports pass or fail, availability and latency as CloudWatch metrics.
- A canary is a script on a schedule, as often as once a minute, hitting the real endpoint through the real auth, network and dependencies, so the feature produces operational metrics even when no subscriber touches it.
- {“Each run publishes SuccessPercent and Duration to the CloudWatchSynthetics namespace to alarm on, sends its logs to CloudWatch Logs, and stores artefacts in S3”=>”screenshots and a HAR file from a browser canary, status codes and timings from an API canary.”}
- For a generative feature the canary sends one fixed golden prompt and asserts on the shape of the answer, the presence of a citation, or a similarity band against a recorded reference, because exact text will not hold; that catches a broken deploy, an expired credential, a regional model outage, or a latency regression before a user finds it.
- A canary can run in a VPC to reach a private endpoint, given either internet access or VPC endpoints for CloudWatch and S3, and runtimes from syn-nodejs-2.0 onwards support active X-Ray tracing, so a failed run has a trace behind it.
- Runs are billed past a free 100 a month, and a canary against a generative endpoint spends Bedrock tokens too, so the interval you set fixes both the bill and the delay before an outage shows up, and the same prompt on a cadence gives a latency baseline.
Pick it when
Pick it for outside-in availability and latency on a low-traffic feature, for a deployment validation smoke test straight after a prompt or model version change, and wherever a synthetic workflow has to keep exercising a journey while traffic is quiet.
It's the wrong answer when
It is the wrong choice for measuring what real users experienced, which needs client-side instrumentation of real sessions; for scoring answer quality across a dataset, which is an evaluation job over a golden set and invocation logs; and for tracing one failed production request, which is X-Ray. A canary is a heartbeat; more than a handful of fixed cases belongs in an evaluation harness instead.