Getting a GenAI Feature to the People Who Use It

The situation

A plant-hire company runs a Bedrock-backed summariser. A depot supervisor’s rough job notes go in, and a structured handover for the next shift comes out. It is one Lambda function calling Converse on a Claude model, with ConverseStream behind a flag nobody has switched on. It has a golden set, a guardrail, and two months of production traffic from an internal script that only the platform team can run.

Two teams now need it reachable by other people. The internal tools team has two front-end developers, a fortnight before the depot rollout, and no backend capacity: they need a chat interface with sign-in, on a URL, in front of forty supervisors. The partner integrations team has three months and a harder constraint. A crew-scheduling vendor will call this service from inside their own product, built by their own engineers on their own release train, and both sides need to start now against something neither can change on its own.

One backend, two asks, and a standing temptation to pick a single delivery mechanism and use it twice.

What actually matters

Read each ask by what it removes. The internal tools team is removing work: hosting, an authentication flow, a chat component, a client for the backend, and the plumbing that keeps one supervisor’s transcript out of another’s. Every hour spent on that is an hour not spent on the depot rollout. The partner team is removing a dependency between two build schedules. Nothing they need is code. What they need is a document precise enough that a vendor’s engineer can write a handler against it and be right the first time. Those are different problems, and the answers have different shapes.

The streaming decision sits upstream of both and cannot be deferred to implementation. A buffered response is one JSON body with a status code, retryable, cacheable, describable in four lines of schema. A streamed response is a sequence of frames over a connection held open for the length of the generation, and it changes the transport, the error model and the timeout at every layer underneath. Once the first token has reached the client there is no status code left to change, so a failure halfway through a generation has to be signalled inside the stream, as data the client is already parsing. That choice propagates into the contract, the component and the gateway configuration alike, and retrofitting it means reworking all three.

Then per-user isolation, which a fortnight makes tempting to defer. Conversation history is personal data with a retention obligation attached, and forty supervisors sharing one table is an incident waiting for an auditor to find it. Whoever builds the interface inherits that obligation, whether they implement the isolation themselves or adopt a framework’s.

Last, the exit. A framework that puts an interface in front of users in a fortnight does so by making decisions on your behalf: a persistence schema, an authorisation model, a transport. Those decisions suit the shape of application the framework was built for, and they hold until the application changes shape. What matters is whether the constraints are visible before you commit, and whether there is a documented drop to the layer beneath when one of them binds.

What we’ll filter on

1. Time to something in front of users. How long from a working model call to a signed-in person typing into it.
2. Decoupling two build schedules. Is there an artefact both teams can build against before either writes a handler?
3. Streaming to the browser. Does incremental delivery survive the whole path, and what configuration does it depend on?
4. Sign-in and per-user history, included. Provided by the option, rather than built.
5. Change without a deployment. Can someone who does not ship code alter the behaviour?
6. The way out. When an opinion binds, is there a documented drop to the layer beneath?

The landscape

Four answers, and what makes the comparison awkward is how little they overlap.

Build the front end yourself

A React application your team owns, talking to an API Gateway REST API in front of the existing function, with Amazon Cognito wired up by hand and the chat component written from scratch. Every decision stays yours: the transport, the transcript schema, the authorisation model, the component library. Incremental delivery works once configured, and the buffered-or-streamed choice stays an explicit one rather than a framework default.

The arithmetic is what rules it out here. Sign-up, sign-in, password reset, token refresh, a message list that renders partial assistant turns as they arrive, resumable conversations, per-user scoping on the transcript table, hosting, and a build pipeline. Each piece is ordinary. Together they are more than two front-end developers finish in two weeks, and none of it is the summariser.

AWS Amplify and its AI kit

Amplify Gen 2 defines a backend in TypeScript under an amplify/ directory, and its AI kit adds two route types to the data schema. A conversation route is a streaming, multi-turn API whose conversations and messages are stored in DynamoDB so a user can resume them. A generation route is a single synchronous request and response, implemented as an AppSync query that returns data shaped by the route’s .returns() definition.

// amplify/data/resource.ts
const schema = a.schema({
  handover: a.conversation({
    aiModel: a.ai.model('Claude 4.5 Haiku'),
    systemPrompt: 'You summarise depot job notes for the next shift.',
    inferenceConfiguration: { temperature: 0.2, maxTokens: 1200 },
  }).authorization((allow) => allow.owner()),

  summariseNotes: a.generation({
    aiModel: a.ai.model('Claude 4.5 Haiku'),
    systemPrompt: 'Return a structured handover.',
  })
    .arguments({ notes: a.string() })
    .returns(a.customType({ risks: a.string().array(), actions: a.string().array() }))
    .authorization((allow) => allow.authenticated()),
});

Underneath a conversation route sit AppSync as the API layer, a Lambda function that loads the history and calls Bedrock’s /converse endpoint, DynamoDB holding the generated Conversation and Message models, and Bedrock serving the model. The kit uses the Converse API throughout, so a model is only usable here if it supports tool use in Converse. a.ai.model() takes friendly names that Amplify keeps in step with Bedrock; an id Amplify has not named yet goes in directly as aiModel: { resourcePath: 'meta.llama3-1-405b-instruct-v1:0' }, which is also the way to point a route at an inference profile. Claude 4.5 and 4.6 models are reached through global inference profile ids for cross-Region routing.

Streaming deserves attention before anyone commits, because it is not HTTP streaming. The Lambda function calls Bedrock with a streaming request, receives chunks, and sends each one to AppSync as a mutation; the browser holds a WebSocket subscription and receives them as they arrive. Two consequences follow. An AppSync subscription message is capped at 240 KB, which no chunk will approach but a full-turn payload might. And the increments travel as GraphQL subscription messages rather than SSE frames, so any consumer that is not an Amplify client has to speak AppSync to read them.

The front end is three imports. createAIHooks(client) returns useAIConversation and useAIGeneration. useAIConversation('handover') names the route from the schema and hands back the messages as React state plus a send handler, updating as chunks land. <AIConversation> from @aws-amplify/ui-react-ai renders them, takes a messageRenderer for markdown, and takes responseComponents, which registers React components as tools the model can invoke by name with typed props. Authentication is defineAuth in the same backend, which provisions the Cognito user pool, and <Authenticator> wraps the component to produce the whole sign-in flow. Amplify Hosting builds the front end and the backend together from a git branch, and npx ampx sandbox gives each developer a disposable stack of their own.

Tools come from the same schema. a.ai.dataTool() points either at a model, where list is the only supported operation, or at a custom query, and the kit describes the parameters to the model, invokes the tool under the caller’s identity so the model sees only that user’s data, and feeds the result back into the turn. A Bedrock knowledge base attaches by adding an AppSync HTTP data source against bedrock-agent-runtime and exposing the retrieve query as a tool, which is a dozen lines rather than a service.

The opinions are where a professional reader should look hardest. Conversation routes support owner-based authorisation only; generation routes support every strategy except owner. A generation route is an AppSync query, so it inherits AppSync’s 30-second request execution ceiling, which is not adjustable, and a long structured generation belongs on a conversation route or outside Amplify altogether. Transcripts live in Amplify’s Conversation and Message models, so a retention rule, a shared team transcript or an export to a warehouse means working around that schema rather than configuring it. The app deploys to one Region, and the model has to be available there.

An OpenAPI document as the contract

Spec-first means the document exists before the handler and is the artefact both organisations agree on. For a GenAI backend the clauses that need settling are the ones a CRUD contract never has to carry: whether the response is buffered or streamed, what a guardrail intervention looks like on the wire, what identifies a generation for later audit, and what a partial result means.

Buffered is the easy half. One application/json response, a schema in components/schemas, and a requestId field tying the response to the invocation log so a complaint three days later is traceable.

Streaming is where the format’s limits show. OpenAPI describes a response body by media type, so a streamed response is one text/event-stream body and the document cannot express the frame sequence as a type. What it can do is name the media type, put a schema for a single event’s data payload in components/schemas, reference it from the operation description, and pin the grammar in examples: which event: names exist, that a terminal event carries the stop reason and the token counts, and that a mid-generation failure arrives as an error event rather than as a status code. That last clause is what stops a vendor’s client treating a truncated stream as a complete answer.

One document then feeds the tooling. A generated client for the vendor, a mock server both sides develop against from day one, a lint pass in CI, and contract tests that run the real handler against the document’s schemas so drift between code and contract fails a build rather than surfacing in integration.

The AWS half is import. SpecRestApi with ApiDefinition.fromAsset makes the document the source of the API rather than a by-product of it, since a SpecRestApi takes all its resources and methods from the file. REST APIs import OpenAPI 2.0 and 3.0; HTTP APIs import 3.0 only. Request validation is configured with x-amazon-apigateway-request-validators, a named map whose entries set validateRequestBody and validateRequestParameters, and applied per method with x-amazon-apigateway-request-validator. Read the limits carefully. API Gateway checks that required parameters in the URI, query string and headers are present and non-blank, and does not check their type or format. It validates the body against a draft-4 JSON schema matched on content type, and performs no validation at all when no content type matches, which is why a data model set to the $default content type is worth having. Responses are never validated. And HTTP APIs have no request validation: import the same document and API Gateway reports info, ignoring the requestBody and schema fields, which is a silent downgrade for anyone who does not read the import output.

A no-code Flow

Amazon Bedrock Flows draws the pipeline as a graph of prompt, condition, knowledge-base and Lambda nodes on a canvas, published as an immutable version with an alias pointing at it. It answers a third question neither team asked: how an operations analyst reorders the steps without waiting on a release. It is not an interface. InvokeFlow still needs something in front of it, and that something is one of the rows above.

Evaluation

Side by side

Every row carries crosses, and no row is close to winning, because the columns are answers to different questions. The two ticks the two teams asked for sit in different rows, which settles the argument about picking one mechanism twice. Three of the marks need their footnotes read out. Amplify’s streaming tick is a subscription over a WebSocket rather than an HTTP stream, so it holds for a browser running the Amplify client and for nothing else. The contract’s streaming tick depends on the integration’s response transfer mode being set to STREAM, since the default is BUFFERED and a route left on it delivers the whole generation at once whatever the function does. The Flow’s cross on streaming is the shape of InvokeFlow, not a configuration anyone can change.

Which ask lands where

The solution

Run both, and keep them apart. The partner contract is written this week and the depot interface is built on Amplify against a conversation route of its own. Neither surface serves the other’s job, and the shared thing underneath them is the model configuration, not the delivery path.

Start with the document, because it unblocks two organisations at once. Two operations: POST /handovers returning application/json for callers that will take a complete answer, and POST /handovers/stream returning text/event-stream for callers that will render progressively. Both take the same request schema. Both return a requestId. The streaming operation’s description fixes the frame grammar and the error event, and the examples show a complete exchange including a guardrail intervention, since that is the case a vendor’s engineer will otherwise guess at. Lint it in CI, publish the mock, and the vendor starts building on day two.

Import that document rather than hand-writing the API. SpecRestApi with ApiDefinition.fromAsset keeps the deployed API and the published contract from drifting, since there is nowhere else for a route to come from. Attach a request validator with validateRequestBody and validateRequestParameters both true, and apply it per method. Then write down what the validator does not do: it checks that required parameters exist, not what they contain, and it never looks at a response. Those promises are held by the contract tests in CI instead, which is the honest division of labour rather than a gap discovered later. Set the streaming route’s integration response transfer mode to STREAM, and leave the buffered route on the default so it keeps caching and response transformation.

The depot interface runs in parallel on Amplify. A conversation route named handover with owner authorisation, defineAuth for the Cognito user pool, <Authenticator> around <AIConversation>, and a data tool pointed at the depot’s equipment model so a supervisor can ask what happened to a machine last week and have the model read only that supervisor’s records. The branch deploys to Amplify Hosting with the backend, and the two front-end developers spend their fortnight on the depot’s actual vocabulary rather than on token refresh.

Keeping one feature behind two doors

The risk in shipping two surfaces is two summarisers. Amplify’s systemPrompt is a string literal in the data schema, which puts the prompt in the front-end repository and ties a wording change to a front-end deployment. If the prompt is already governed in Bedrock Prompt Management, resolve it in a custom conversation handler so both doors read the same version, and pin the same model id in both places. Ship the prompt version, the model id and the guardrail id as one bundle, as the release pipeline already does for the service. The alternative is two prompts drifting for a quarter and an evaluation score that only describes one of them.

When Amplify’s opinions bind

Name the triggers before the rollout, because leaving late means migrating live transcripts. There are four. An authorisation model other than owner on the conversation route, which the route does not support. A second consumer that is not a browser running the Amplify client, since the increments are AppSync subscription messages. A retention, sharing or export rule that Amplify’s Conversation and Message models do not express. And a structured generation that needs longer than AppSync’s 30-second request execution ceiling.

The way down is graded. First, a custom conversation handler: instantiate ConversationHandlerFunction from @aws-amplify/ai-constructs/conversation with your own entry and models, and reference it from the route, which keeps the routes, the auth and the components while you own the turn logic. Below that, the CDK constructs under backend.data.resources are reachable directly, so the AppSync API can take data sources and resolvers Amplify never generated. Below that, the API the partner team has already built is a complete second delivery path, and moving the depot UI onto it becomes a front-end change rather than a rebuild. Writing that ladder down during the fortnight is what stops the fortnight’s decision becoming permanent by accident.

Worked example

The streaming operation, cut down to the clauses that decide something:

/handovers/stream:
  post:
    operationId: streamHandover
    x-amazon-apigateway-request-validator: body-and-params
    requestBody:
      required: true
      content:
        application/json:
          schema: { $ref: '#/components/schemas/HandoverRequest' }
    responses:
      '200':
        description: >
          Server-sent events. Frames are `event: delta` carrying
          HandoverDelta, then exactly one terminal frame: `event: done`
          carrying stopReason and usage, or `event: error` carrying a
          code. A stream that ends without a terminal frame is
          incomplete and must not be treated as an answer.
        content:
          text/event-stream:
            schema: { type: string }
            examples:
              guardrailIntervention:
                $ref: '#/components/examples/GuardrailStream'

The text/event-stream schema is string because that is all OpenAPI can say about a frame sequence; HandoverDelta and the terminal frames are real schemas in components, referenced from the description and pinned by the examples. A generated client will not enforce them, so the contract tests do.

The validator sits beside the paths, and the same file carries the integration:

x-amazon-apigateway-request-validators:
  body-and-params:
    validateRequestBody: true
    validateRequestParameters: true

Two weeks later the depot rollout goes out on Amplify, forty supervisors sign in through Cognito, and the vendor’s client is passing contract tests against a handler nobody has deployed yet. The summariser itself was never touched.

What’s worth remembering

1. AWS Amplify’s AI kit turns a.conversation() and a.generation() in the data schema into AppSync plus a Lambda calling Bedrock’s Converse API plus DynamoDB, with useAIConversation and <AIConversation> binding a component to it and defineAuth supplying Cognito.
2. Amplify streams to the browser over an AppSync WebSocket subscription rather than as an HTTP stream, so incremental delivery reaches an Amplify client and nothing else.
3. Amplify’s binding constraints are owner-only authorisation on conversation routes, AppSync’s 30-second ceiling on a generation route, and its own Conversation and Message schema; the graded exit is a custom ConversationHandlerFunction, then the CDK constructs beneath it.
4. An OpenAPI document decouples two teams’ schedules by existing before either handler does, and for a streamed response it can fix the media type, the event schemas and the terminal-frame grammar, but not the frame sequence as a type.
5. API Gateway request validation checks that required parameters are present, not their type or format, validates bodies against a draft-4 schema per content type, never validates responses, and is absent entirely from HTTP APIs, which ignore requestBody on import.
6. A chat interface, an API contract and a no-code Flow answer three different questions, and the cost of using one of them for all three is measured in quarters.