How to Manage Prompts Across Thirty Services on Bedrock

The situation

The platform team owns Bedrock access for the whole company. Roughly thirty services, a support assistant, a ticket classifier, a marketing-copy drafter, a translation pipeline, a meeting summariser, and twenty-five others, call Bedrock in production, each with its own prompt. The prompts were drafted separately by product teams, copied between codebases, embedded as string literals, sometimes templated with f-strings, sometimes loaded from a Markdown file.

What happened last quarter is going to happen again. A product engineer tweaked the meeting-summariser’s system promptSystem promptThe instruction block that frames the model’s behaviour for a session, separate from the user’s messages., changed “concise” to “brief” in what looked like a clean-up, deployed to production, and retention on the daily summary email dropped 12% for eight days before anyone correlated the code change. The prompt had no version history the product team could see. The A/B infrastructure had no concept of a prompt as something to vary. The monitoring dashboard reported Bedrock latency and error rate; it didn’t report whether the output was any good.

Platform’s ask: a prompt management story for the whole company. Version prompts, test them before release, roll them out alongside the code that calls them (or independently, if that’s better), measure their impact, and stop letting string literals in thirty repos be the authoritative copy.

What actually matters

A prompt is the text that goes into the modelModelA trained set of weights plus the architecture that makes them useful – the thing you load up and run inference against. ahead of any user input. It is, functionally, configuration: it changes behaviour, it’s smaller than code, it needs review, and it needs versioning. The failure modes are the ones every config-management practice was invented to address, drift, shadow copies, untested change, silent regression, no rollback.

The first decision is where prompts live as source of truth. Checked into the service repo? Central repo? A managed Bedrock resource? A database?

The second is how they’re versioned. Git commit hashes? Semantic versions? Bedrock prompt versions? All of the above, coordinated?

The third is how they’re released. Deployed with the code that uses them, or independently? Is a prompt change a deployment, a feature flag flip, or a config push?

The fourth is how they’re tested. Before the change hits production, somebody runs the new prompt against a bank of examples and checks the outputs. Is that bank owned by the prompt author? The product team? Platform?

The fifth is how they’re parameterised. A prompt usually has slots, user input, retrieved context, session state. The templating language matters: f-strings lose their context when you refactor; Jinja gains power but adds a dependency; a simple {variable} substitution is predictable. Managed registries usually have their own template syntax to learn.

The sixth is how they’re attributed. When one prompt feeds thirty services, the bill, the latency, and the quality signal have to be broken down by caller, otherwise the platform team can’t tell which service is driving which problem.

The seventh is ownership: who edits the prompt, who approves the edit, who rolls it back? Without a clear answer, every service’s prompt is owned by the last engineer who touched it, which is to say, owned by no one.

What we’ll filter on

1. Source-of-truth clarity, one place, many places, or a registry?
2. Versioning and rollback, immutable versions, diffs, easy revert?
3. Deployment shape, bundled with code, pushed independently, feature-flagged?
4. Evaluation coverage, tests run before a prompt ships?
5. Per-caller attribution, cost, latency, quality broken down by service?

The landscape

Bedrock Prompt Management. AWS-native prompt registry. Create a prompt with a template, variables in double braces, a model or inference profile to run it on, the four base inference parameters (maxTokens, stopSequences, temperatureTemperatureA knob (usually 0 to 2) that controls how much the model deviates from its highest-probability next token. and topP), and, on the CHAT template type, a system prompt and prior turns. The working draft is mutable; CreatePromptVersion freezes a numbered snapshot, template, variables, model choice, and inference config locked together. Versions number from 1, and ten per prompt is a fixed quota in each Region. The account quota, 500 prompts per Region, is the adjustable one. There are no aliases: nothing inside Bedrock points production at version 12; a version is referenced by its ARN, and that reference lives wherever you put it. To invoke, pass the version ARN as the modelId to Converse or ConverseStream with a promptVariables map, and Bedrock renders the template, runs the model, returns the response. InvokeModel accepts the same ARN, but only for a prompt whose configured model is an Anthropic Claude or a Meta Llama. IAM scopes who can create, version, and invoke prompts. Covers 2 cleanly and 1 partially; nothing for 3, 4, and 5, so the routing and measurement layers are ours to build.

Git-backed templates in a shared repo. A prompts/ directory in a shared repo, one file per prompt, Jinja2 or Handlebars or a plain-text template with named placeholders. A small library in each service loads the prompt, substitutes variables, calls Bedrock. Versioning is Git commits; releases are tags; tests sit next to the templates in CI. Nothing AWS-specific; works identically if Bedrock moves to a different model. Ticks 1, 2, 3, 4 cleanly; 5 depends on observability we add.

Parameterised prompts in each service’s config. Prompts live in each service’s config file (YAML, JSON), deployed with the service, versioned with the service. Least work to set up; the baseline thirty-services-each-doing-their-own-thing pattern, formalised. Ticks 3 cleanly; fails 1 and 5.

LangChain’s PromptTemplate + LangSmith. Prompts as code in a shared Python package; LangSmith as the evaluation and observability surface. Prompts versioned in the package, evaluated with LangSmith datasets, observed per-invocation. Strong on 4 and 5; separate SaaS; tied to LangChain’s abstractions.

A prompt database. A DynamoDB or Postgres table, or SSM Parameter Store itself, holding prompt bodies, versions, and metadata. Services fetch the active prompt at call time. Flexible, but puts prompt changes one write away from production, fast, and dangerous without a deployment gate.

Hybrid. Git + Bedrock Prompt Management + a routing parameter. The pattern most platform teams land on. Prompts authored in Git, reviewed in PRs, evaluated in CI. On merge, a pipeline calls CreatePromptVersion; the new version’s ARN is the release artefact. A Parameter Store parameter per prompt per stage (/prompts/summariser/production) holds the ARN currently in service; promotion and rollback are parameter writes. Git is the source; Bedrock is the immutable registry; Parameter Store is the alias layer Bedrock doesn’t ship.

Evaluation

Side by side

Option	Source of truth	Versioning	Deployment	Evaluation	Attribution
Bedrock Prompt Management	Bedrock resource	Numbered versions, 10 max	API call	Manual / custom	Request metadata in logs
Git-backed templates	Repo	Commits, tags	With service	CI-driven	Build it ourselves
Per-service config	Each service	With service	With service	Each team’s job	None central
LangChain + LangSmith	Python package	Package versions	With service	LangSmith datasets	LangSmith traces
Prompt database	DB rows	Row versions	DB write	Optional	Depends
Git + Bedrock + SSM (hybrid)	Git, with mirror	Git commits → Bedrock versions	Pipeline + parameter flip	CI + golden set	Invocation logs + caller metrics

For a platform team with 30 callers, the hybrid wins on the trade-offs. Git carries the authoring workflow; Bedrock Prompt Management carries the immutable version registry; Parameter Store carries the routing, which version each stage is actually serving. No single piece hits all five attributes. Prompt Management covers versioning, Parameter Store covers routing, and attribution is assembled on top of both.

The prompt lifecycle, end to end

The solution

Authoring. Prompts live in prompts/ in a shared repo. Each prompt is a Jinja2 template plus a YAML sidecar with the inference config (temperature, top-p, max tokens, stop sequences), the intended foundation model, and ownership metadata (team, primary contact, service list). PRs require review by the prompt owner; SMEs are added as reviewers via CODEOWNERS based on domain.

Evaluation in CI. A GitHub Action runs on every PR. For each changed prompt, it loads a 50-example smoke set (small, fast, runs in under two minutes), invokes the model with the new template, and scores with a mix of reference metrics (BLEU, ROUGE, exact-match for structured outputs) and LLM-as-judge (another Claude call scoring each output 1-5 on defined rubrics). Thresholds are per-prompt, the summariser has different quality criteria than the ticket classifier. A regression blocks merge.

Release pipeline. On merge to main, a pipeline loops through changed prompts and calls CreatePromptVersion on Bedrock. The version is an immutable snapshot, template, variables, model choice, and inference config frozen together, and its ARN is the release artefact. Two quotas shape the loop. CreatePromptVersion is capped at two requests a second per Region, so the pipeline works through the changed prompts in series rather than fanning them out. And a prompt holds ten versions, a figure that cannot be raised. A weekly edit reaches the cap inside three months, so before each release the pipeline deletes the oldest version with DeletePrompt and a promptVersion, having first confirmed no stage parameter still points at it. Git keeps the full history either way. The pipeline then reruns the evaluation harness against a larger golden set (500-2000 examples), invoking the new version’s ARN directly; same judges as CI, bigger net. If the scores are within tolerance of the version currently serving production, the pipeline writes the new ARN into the prompt’s staging parameter and the canary starts at 5% of traffic. The platform SDK does the routing, resolving the candidate parameter for canary callers and the production parameter for everyone else. 24 hours of metrics; if error rate and user-facing quality signals hold, the pipeline writes the production parameter.

Approval. The step between staging and production is an approval workflow, and it gates on two conditions rather than one signature. The golden-set score for the candidate version has to clear the threshold recorded in that prompt’s sidecar, and the named owner for that prompt has to approve the version by name. The signature only means something when it sits on a number somebody agreed in advance. A reviewer clicking approve on a version that scored below baseline is a rubber stamp with an audit trail attached. If the owner is on leave, the deputy named in the sidecar approves; if neither is available, the version waits. A prompt nobody owns is how the summariser regression happened.

Runtime. The alias layer is ours, not Bedrock’s. One Parameter Store parameter per prompt per stage, /prompts/summariser/production, holds the version ARN currently in service. The platform SDK resolves the parameter and caches it with a short TTL, because Parameter Store reads default to 40 transactions a second across GetParameter, GetParameters and GetParametersByPath combined, and thirty services would spend that ceiling on lookups before they invoked anything. Higher throughput lifts GetParameter to 10,000 a second for an extra charge; a cache avoids the bill. The SDK then calls Converse with the version ARN as the modelId and a promptVariables map. Bedrock renders the frozen template, runs the model, returns the response. A call that names a managed prompt cannot also carry system, inferenceConfig, toolConfig or additionalModelRequestFields, which is the behaviour we want: those are settled in the version. Services never embed a version number; they embed a parameter name.

Templates that live outside Prompt Management. Three of the thirty services render their prompts in their own process, either because they call a model outside Bedrock or because their account has no Prompt Management access at all. For those, the pipeline uses Amazon S3 to store template repositories: one prefix per prompt, one object per version, the same YAML sidecar beside each template. Object versioning is on, so an accidental overwrite is recoverable rather than a rewrite of history. The bucket policy grants the runtime roles s3:GetObject and nothing else; writes belong to the pipeline role alone. A service reads the template it was given and cannot edit the copy every other service reads. That is the property CreatePromptVersion gives us inside Bedrock, built out of a bucket policy instead.

Rollback. One parameter write: point /prompts/summariser/production back at the previous version’s ARN. No service redeploy; the change propagates as SDK caches expire, seconds to a minute. Parameter Store keeps its own version history per parameter, up to 100 versions before the oldest drops off, so the rollback is itself audited, who repointed what, when, to which ARN. The revert button exists, it takes seconds, and it leaves a paper trail. Bedrock ships no such button; a parameter per stage builds one out of parts the platform team already runs.

Per-caller attribution. Bedrock has a hook for this, so the platform SDK uses it rather than inventing one. Converse takes a requestMetadata field and InvokeModel an X-Amzn-Bedrock-Request-Metadata header, each holding up to 16 key-value pairs of at most 256 characters, and those pairs land in the model invocation log beside the input and output token counts. The platform SDK sets caller ID, prompt name and version on every call. Model invocation logging is off until somebody turns it on, and once on it writes request and response bodies inline up to 100 KB, with anything larger going to an S3 bucket under a data prefix. The log record also carries identity.arn automatically, so a call that omits the metadata is still traceable to a role. The SDK emits the same three dimensions as a custom CloudWatch metric, because a dashboard reads a metric faster than it scans a log group. When one service reports that the summariser is slow, platform can see whether it’s slow for everyone or only for them, and if only for them, which argument shape goes with the slow calls.

Catching regression after release. CI and the golden set only cover the examples somebody thought to write down, so two checks run against live traffic as well. The first samples production. A Lambda function takes a small percentage of responses and asserts the shape the caller depends on: valid JSON, the required fields present, length inside the band the prompt specifies. The pass rate goes out as a custom CloudWatch metric dimensioned by prompt and version, and an alarm fires when it sits below its floor for two periods running. Regression then shows up between releases, not only at one. The second check runs on publication. A Step Functions state machine walks a fixed set of edge cases against each new version ARN: empty input, a 40,000-tokenTokenThe unit of text an LLM actually sees – usually a short character sequence, not a whole word. transcript, a meeting held in Portuguese, a recording where one person talks for the whole hour. It fails the execution on the first output that breaks. A state machine keeps the awkward inputs in one place, with per-case results and retries, rather than scattered through a test file that times out on the long transcript.

Audit and access. Two logs answer two different questions. CloudTrail covers the registry: CreatePrompt, CreatePromptVersion, UpdatePrompt and DeletePrompt are management events, recorded by default with the principal, the timestamp and the request parameters, so “who changed the summariser prompt, and when” is a CloudTrail query. CloudTrail reaches the runtime too, but only once configured for it. When an invocation names a managed prompt, Bedrock performs RenderPrompt, a permission-only action that surfaces as a CloudTrail data event on the AWS::Bedrock::Prompt resource type. Data events are off by default and billed separately, so the advanced event selector has to be added deliberately. The application’s own CloudWatch Logs line is still worth writing: one structured line per invocation carrying the prompt identifier, the version ARN it resolved, the caller and the request ID, which is what joins a support ticket to a release without paying for data events on every call. When support asks why one user got a mangled summary on Tuesday, the log line names the version and the trail names the engineer who published it.

Worked example

Someone opens a PR changing “concise” to “brief” in the summariser prompt.

1. CI runs the 50-example smoke set. The LLM-as-judge rubric includes a “length appropriateness” criterion. The new prompt scores 3.2/5 on that criterion vs the baseline’s 4.1/5, outputs are now shorter than the ideal. CI posts the regression; reviewer asks “was that intentional?”
2. Author decides the intent was wording cleanup, not behaviour change. They revert. Incident prevented in three minutes.

Alternative reality: author insists. Reviewer approves. PR merges.

1. Pipeline prunes the oldest stored version, then freezes summariser version 7 with CreatePromptVersion. The golden-set evalBenchmarkA standardised test set used to score and compare models. runs the 500-example set against the new version’s ARN. Overall quality score holds, but the length-appropriateness sub-metric is down. Platform’s quality dashboard flags the change for human review before the staging parameter moves.
2. Product team decides “shorter is fine” and approves. The staging parameter repoints; canary starts at 5%.
3. User-facing retention metric in Datadog is wired into the canary gate. 24 hours in, retention on the summariser’s daily email has dipped 4% on the canary users; p<0.01. Pipeline aborts the canary. The production parameter stays on version 6’s ARN.
4. Rollback is automatic; no action required. Author sees the abort notification and has data to work with.

What didn’t happen: eight days of silent regression, a confused postmortem, and a product team blaming engineering.

What’s worth remembering

1. Prompts are configuration. Treat them like it: version control, review, CI, release pipeline, rollback. A prompt change takes seconds to make and days to detect when it goes wrong.
2. Git is the source of truth; Bedrock is the runtime registry. Git gives PRs, diffs, code review, and CI; Bedrock gives numbered versions referenced by ARN; Parameter Store gives the pointer that says which ARN is live.
3. A prompt version is invoked by ARN. Pass it as the modelId with a promptVariables map and the template renders server-side, inference config included. Converse takes any text model the Converse API supports; InvokeModel takes a managed prompt only when the version names an Anthropic Claude or Meta Llama model.
4. Prompt Management has no alias, and ten versions per prompt is a fixed quota, so build both the pointer and the pruning. An SSM parameter per prompt per stage holds the live version ARN; rollback is repointing the parameter, audited through parameter history, with no thirty-service redeploy.
5. Two logs, two questions. CloudTrail management events record who created, updated or deleted a prompt version, and a data event selector on AWS::Bedrock::Prompt records RenderPrompt at invocation time; the application’s own line adds the caller and the parameter it resolved.
6. Don’t let prompts live as string literals in thirty repos. Scattered copies are the root cause behind most prompt regressions that reach production.

One prompt, thirty callers, a versioned registry, a release pipeline, and a rollback that lands faster than the Slack thread asking “did we change something?” The service owners still own their prompts; the platform team stopped letting them own them badly.