Exam-style
A nightly enrichment batch calls Amazon Bedrock once per document. Since the retrieval step started attaching more passages to each prompt, the job has gone from forty minutes to six hours and still ends with thousands of failures. Every call is wrapped in a retry-with-backoff helper that catches any exception and tries five times. The failure log holds a mix of `ThrottlingException` and `ValidationException`. What should change first?
Reveal the answer
E. Split the handler by exception name: back off on `ThrottlingException`, and fail `ValidationException` immediately onto a repair path
The two names describe different faults, and one helper treating them alike is what turned forty minutes into six hours. ValidationException means the request itself is wrong, so the identical request is rejected identically at attempt one and attempt ten thousand; here the retrieved passages grew until the assembled prompt exceeded the model’s context window, a content handling defect fixed by trimming or re-chunking the input, not by waiting. Each failed document then adds four more round trips and a full backoff ladder before reaching the same rejection, which is most of the five hours and twenty minutes the job gained. ThrottlingException is the only one of the two that backoff helps, because the request is fine and the account is over its per-model quota. Retryable names to recall: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Names to surface rather than retry: ValidationException, AccessDeniedException, ResourceNotFoundException. ServiceQuotaExceededException sits apart: a 400 against an account quota, which a later run can clear and a backoff loop cannot. Of the distractors: a higher retry count lengthens the ladder that is already the problem; a quota increase and Provisioned Throughput both address the throttling half and leave every oversized prompt failing exactly as before; and a larger context window delays the same failure, since nothing in the pipeline caps how many passages get attached.
Q. A batch job wrapped in a blanket retry now takes six hours and still fails. The logs hold ThrottlingException and ValidationException. What changes first?
A. Handle them separately. ThrottlingException means the account went over a Bedrock quota, and backoff with jitter clears a per-model tokens-per-minute one; the per-account tokens-per-day ceiling returns the same name and clears only the next day. ValidationException is the caller’s fault, here prompts that outgrew the Context windowThe maximum number of tokens an LLM can attend to in a single call – prompt plus output combined. as the retrieved passages piled up, so it must fail fast onto a repair path that trims or re-chunks the input and re-queues the document. A Converse call can return 200 with the stop reason model_context_window_exceeded, which an error-only handler misses.
Why? Retry decisions come from the exception name, not the call site. Retryable: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Surface instead: ValidationException, AccessDeniedException, ResourceNotFoundException. ServiceQuotaExceededException arrives as a 400 against an account quota, so defer the work and raise the quota rather than looping. A helper that catches everything throws that name away, which is how a permanent error becomes a slow permanent error and a quota ceiling becomes a self-inflicted outage. Log the exception name, the request ID and the assembled token count on every failure, so the next six-hour run is diagnosable in minutes.