Exam-style
A nightly enrichment batch calls Amazon Bedrock once per document. Since the retrieval step started attaching more passages to each prompt, the job has gone from forty minutes to six hours and still ends with thousands of failures. Every call is wrapped in a retry-with-backoff helper that catches any exception and tries five times. The failure log holds a mix of `ThrottlingException` and `ValidationException`. What should change first?
Reveal the answer
E. Split the handler by exception name: back off on `ThrottlingException`, and fail `ValidationException` immediately onto a repair path
The two names describe different faults, and one helper treating them alike is what turned forty minutes into six. ValidationException says the request itself is wrong, so the identical request will be rejected identically at attempt one and attempt ten thousand; here the retrieved passages grew until the assembled prompt exceeded the model’s context window, which is a content handling defect fixed by trimming or re-chunking the input, not by waiting. Retrying it burns five round trips and the whole backoff ladder per document to reach the same rejection, which is most of the added five hours. ThrottlingException is the only one of the two that backoff helps, because the request is fine and the account is being paced. Retryable names to recall: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Names to surface rather than retry: ValidationException, AccessDeniedException, ResourceNotFoundException, ServiceQuotaExceededException. Of the distractors: a higher retry count lengthens the ladder that is already the problem; a quota increase and provisioned throughput both address the throttling half and leave every oversized prompt failing exactly as before; and a larger context window hides a prompt that will keep growing, since nothing in the pipeline caps how many passages get attached.
Q. A batch job wrapped in a blanket retry now takes six hours and still fails. The logs hold ThrottlingException and ValidationException. What changes first?
A. Handle them separately. ThrottlingException is transient and paced, so exponential backoff with jitter is the right response. ValidationException is the caller’s fault, here prompts that outgrew the Context windowThe maximum number of tokens an LLM can attend to in a single call – prompt plus output combined. as the retrieved passages piled up, so it must fail fast onto a repair path that trims or re-chunks the input and re-queues the document.
Why? Retry decisions come from the exception name, not the call site. Retryable: ThrottlingException, ModelNotReadyException, ModelTimeoutException, InternalServerException, ServiceUnavailableException. Surface instead: ValidationException, AccessDeniedException, ResourceNotFoundException, ServiceQuotaExceededException. A helper that catches everything throws that name away, which is how a permanent error becomes a slow permanent error and a quota ceiling becomes a self-inflicted outage. Log the exception name, the request ID and the assembled token count on every failure, so the next six-hour run is diagnosable in minutes.