Exam-style
A nightly enrichment batch calls Amazon Bedrock once per document. Since the retrieval step started attaching more passages to each prompt, the job has gone from forty minutes to six hours and still ends with thousands of failures. Every call is wrapped in a retry-with-backoff helper that catches any exception and tries five times. The failure log holds a mix of `ThrottlingException` and `ValidationException`. What should change first?
Reveal the answer
E. Split the handler by exception name: back off on `ThrottlingException`, and fail `ValidationException` immediately onto a repair path
The two names describe different faults, and one helper treating them alike is what turned forty minutes into six hours. ValidationException means the request itself is wrong, so the identical request is rejected identically at attempt one and attempt ten thousand; here the retrieved passages grew until the assembled prompt exceeded the model’s context window, a content handling defect fixed by trimming or re-chunking the input, not by waiting. Each failed document then adds five round trips and a full backoff ladder before reaching the same rejection, which is most of the added five hours. ThrottlingException is the only one of the two that backoff helps, because the request is fine and the account is over its per-model quota. Retryable names to recall: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Names to surface rather than retry: ValidationException, AccessDeniedException, ResourceNotFoundException. ServiceQuotaExceededException sits apart: a 400 against an account quota, which a later run can clear and a backoff loop cannot. Of the distractors: a higher retry count lengthens the ladder that is already the problem; a quota increase and Provisioned Throughput both address the throttling half and leave every oversized prompt failing exactly as before; and a larger context window delays the same failure, since nothing in the pipeline caps how many passages get attached.
Q. A batch job wrapped in a blanket retry now takes six hours and still fails. The logs hold ThrottlingException and ValidationException. What changes first?
A. Handle them separately. ThrottlingException is transient, raised when the call exceeds an account quota for the model, so exponential backoff with jitter is the right response. ValidationException is the caller’s fault, here prompts that outgrew the Context windowThe maximum number of tokens an LLM can attend to in a single call – prompt plus output combined. as the retrieved passages piled up, so it must fail fast onto a repair path that trims or re-chunks the input and re-queues the document.
Why? Retry decisions come from the exception name, not the call site. Retryable: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Surface instead: ValidationException, AccessDeniedException, ResourceNotFoundException. ServiceQuotaExceededException arrives as a 400 against an account quota, so defer the work and raise the quota rather than looping. A helper that catches everything throws that name away, which is how a permanent error becomes a slow permanent error and a quota ceiling becomes a self-inflicted outage. Log the exception name, the request ID and the assembled token count on every failure, so the next six-hour run is diagnosable in minutes.