The situation
A platform team operates a customer managed KMS key in a shared-services account (999999999999), used to wrap the envelope keys that protect inbound event payloads. Twelve spoke accounts host an event-processing Lambda fleet. Each Lambda is triggered by EventBridge or SQS, one message per invocation; it assumes an execution role (lambda.amazonaws.com with a per-account EventProcessorRole); it calls kms:Decrypt once to unwrap the payload envelope, processes the plaintext, and exits. It lives for under a second in the typical case and never longer than the function timeout.
Security’s constraints follow the shape of the workload. Permission is ephemeral, per execution. Each permission issued leaves its own audit evidence. Scope is Decrypt on exactly this envelope. And if a spoke account is compromised, withdrawal happens without a policy rewrite.
What actually matters
The natural instinct, “put it in a policy”, doesn’t fit this workload. A key policy statement allowing twelve spoke roles to call kms:Decrypt on the shared key would work. It would also mean that, at any moment, any principal in any of those twelve accounts that could assume the spoke role could decrypt any envelope ever wrapped with that key. The workload is decrypting one envelope, on one execution, for under a second, once. The policy instrument has an effective lifetime measured in months; the permission the workload needs has one measured in milliseconds. That mismatch is where blast radius expands, and it also makes withdrawal expensive: taking a compromised spoke out of a key-policy model is a PutKeyPolicy edit with all the ceremony that entails. What we want instead withdraws in one administrative action bounded to the live permissions, leaving the key policy and twelve IAM policies untouched.
With a broad key-policy grant, every Decrypt call lands in CloudTrail as a single event: this role decrypted something at this time. To prove that permission to decrypt envelope evt-abc123 was issued, used, and ended, we would have to correlate that Decrypt with some other event carrying the envelope’s identity. What we want instead leaves its own creation record and its own termination record, joinable by id, with the envelope’s encryption context on the use in between. Three joined records, where the policy model gives one.
Nobody asked for encryption-context scoping, and it is still worth having. A Decrypt request against a symmetric encryption key carries additional authenticated data, the encryption context, and KMS can require a match on it as a precondition of the permission. A permission bound to {envelope-id=evt-abc123, tenant=acme} does not work against any other envelope, so a credential intercepted in flight cannot be replayed against a different payload. Asymmetric and HMAC keys take no encryption context, so the scoping exists only on symmetric encryption keys. This one is.
A KMS authorisation created at t=0 is not immediately in effect across all of KMS. AWS says the change typically propagates in less than a few seconds, usually within five minutes, and in some cases several minutes. It publishes no hard ceiling. A sub-second handler loses that race, so the answer has to include a mechanism that lets a handler use a freshly created permission before propagation finishes.
What we’ll filter on
- Per-execution lifetime. Permission exists for one handler run and then ceases.
- Per-execution audit. Each issuance leaves a distinct CloudTrail record.
- Operation scope =
Decrypt. No Encrypt, no GenerateDataKey, no ReEncrypt, just the one operation. - Envelope-context scoping. The permission works for one envelope, not any envelope.
- Surgical revocation. Remove permission without editing the key policy or redeploying anything.
The landscape
A kms:Decrypt call is evaluated against six instruments, and they do not all do the same kind of work. Three can allow: the key policy, an IAM policy in the caller’s account, and a grant. An allow from any one of the three is sufficient, and the absence of an allow in all three is itself a denial. Three can only restrict: a VPC endpoint policy, a service control policy, and a resource control policy. An explicit deny in any of the six ends the request.
1. The key policy. Every KMS key has exactly one, and it is mandatory. Key policies are Regional. No AWS principal, including the account root user and the key’s creator, has any permission on the key unless it is explicitly allowed, and never denied, in a key policy, an IAM policy, or a grant. The route through IAM opens only when the key policy delegates to IAM in a statement whose principal is the account. Without that statement an IAM allow has no effect on the key, though an IAM deny still does.
2. IAM identity policies in the caller’s account. The IAM policies that govern a principal’s use of a key are always defined in that principal’s own account, including when the key belongs to another one. Cross-account access through the key policy needs both halves: the key policy in the shared-services account names the external account, and an IAM policy in the external account delegates the actions to a role. The key policy sets what the external account may be given; the IAM policy decides who in it gets it.
3. Grants. A separate allow-only instrument attached to one key, naming one grantee, one operations list, and optional constraints. Created by CreateGrant. AWS describes a grant as an alternative to the key policy rather than an addition to it: a grant can authorise a principal the key policy never names.
4. VPC endpoint policies. If the Lambda reaches KMS through an interface VPC endpoint, the endpoint policy narrows which keys and principals may use that path. It allows nothing on its own.
5. Service control policies. An SCP follows the caller, so the spoke account’s SCP is the one in scope for the Decrypt call. SCPs never grant, they have no effect in the management account, and they do not restrict a service-linked role.
6. Resource control policies. KMS is one of the services RCPs cover. An RCP follows the resource, so the one in scope is attached to the shared-services account that owns the key, and it applies even to callers outside the organisation. RCPs never grant either, they have no effect on AWS managed keys, and kms:RetireGrant is specifically outside what they can restrict.
Evaluation
Side by side
| Instrument | Per-execution lifetime | Per-execution audit | Scope = Decrypt only | Context-bound | Surgical revocation |
|---|---|---|---|---|---|
| Key policy | ✗ | ✗ | ✗ | ✗ | ✗ |
| IAM identity policy | ✗ | ✗ | ✗ | ✗ | ✗ |
| Grants | ✓ | ✓ | ✓ | ✓ | ✓ |
| VPC endpoint policy | ✗ | ✗ | ✗ | ✗ | ✗ |
| Service control policy | ✗ | ✗ | ✗ | ✗ | ✗ |
| Resource control policy | ✗ | ✗ | ✗ | ✗ | ✗ |
Key policies and IAM policies are persistent JSON. Editing either one per execution is not workable, neither emits an event that says a permission was issued, and withdrawal means a policy edit. Endpoint policies, SCPs and RCPs are not issuance mechanisms at all: they narrow and they deny. Grants are the one instrument designed to be created and deleted on the timescale of a single operation. Each one is scoped to one grantee, one key and a fixed operations list, and each gets its own CloudTrail entries for creation and termination. The key policy still has to authorise the CreateGrant call, but the per-execution permission lives in the grant.
Grant lifecycle
The solution
A grant is a KMS resource identified by a GrantId of 1 to 128 characters. CreateGrant is the only operation that returns a GrantToken, a nonsecret base64 string of 1 to 8192 characters. Every call returns a fresh token, even when Name makes a retry hand back the original GrantId. One customer managed key holds up to 50,000 grants, an adjustable quota that integrated AWS services draw on too. It has to be a customer managed key here: an AWS managed key cannot be used in cryptographic operations directly, and its key policy cannot be edited.
Shape of a grant. KeyId names the one key the grant applies to. GranteePrincipal is the identity that receives the permissions, and it has to be an AWS principal: an account, an IAM user, an IAM role, a federated user, or an assumed-role user. It cannot be a service principal, an IAM group, or an organisation, and GranteeServicePrincipal is the separate parameter for a service, which also requires a SourceArn constraint and a retiring principal. Operations comes from a fixed list of seventeen (Decrypt, Encrypt, GenerateDataKey, GenerateDataKeyWithoutPlaintext, GenerateDataKeyPair, GenerateDataKeyPairWithoutPlaintext, ReEncryptFrom, ReEncryptTo, Sign, Verify, GenerateMac, VerifyMac, DeriveSharedSecret, GetPublicKey, DescribeKey, CreateGrant, RetireGrant), and anything outside it, or unsupported by the key spec, returns a ValidationError. Key management operations such as ScheduleKeyDeletion and PutKeyPolicy are not on it. RetiringPrincipal is optional and names who may retire the grant. Name is optional, makes a retried CreateGrant idempotent, and cannot be used to identify the grant in any other operation.
Constraints. There are two kinds. An encryption-context constraint is EncryptionContextEquals for an exact case-sensitive match on the request’s pairs, or EncryptionContextSubset for “the request must include these pairs and may carry others”. It is valid only on a grant for a symmetric encryption key, takes up to eight pairs, and caps each value at 384 characters. SourceArn restricts the grant to requests made on behalf of one AWS resource, works on grants for every key type, and is evaluated only when an AWS service makes the request. A grant can carry SourceArn alongside one encryption-context form, but never both encryption-context forms. Constraint values and the grant name can appear in plaintext in CloudTrail, so neither holds anything sensitive.
Allow-only, and no expiry. Grants cannot deny; deny logic lives in key policies, IAM policies, SCPs, RCPs or endpoint policies. CreateGrant also takes no TTL. A grant exists until something deletes it, so per-execution lifetime is a property of the caller’s code, not of the grant.
The key policy still authorises the creation. CreateGrant is itself a KMS operation, and the caller needs kms:CreateGrant on the key. For a cross-account call the key policy must permit the external principal to call it, and IAM in the caller’s account must allow it too. Once the grant exists, the grantee needs nothing from the key policy for the operations it lists.
kms:CreateGrant is close to kms:PutKeyPolicy in strength. A principal whose CreateGrant permission comes from a policy can create a grant for any grant operation on that key, including operations the principal does not itself hold. The narrower rule, that a child grant may allow only a subset of its parent’s operations with constraints at least as strict, binds only a principal whose CreateGrant permission came from a grant. An unconstrained kms:CreateGrant on this key is therefore close to full use of it, which is why the condition keys below matter.
Condition keys for CreateGrant. kms:GranteePrincipal fixes who may be named as grantee, kms:RetiringPrincipal who may be named as retiring principal, and kms:GranteeServicePrincipal and kms:RetiringServicePrincipal do the same for service grants. kms:GrantOperations fixes which operations a grant may permit; it is multi-valued, so it needs a set operator (ForAllValues:StringEquals) to mean “every operation in the request is in my list”. kms:GrantConstraintType requires an encryption-context constraint of a named type, EncryptionContextEquals or EncryptionContextSubset, and kms:GrantConstraintSourceArn matches the SourceArn value. kms:GrantIsForAWSResource allows the call only when an integrated AWS service makes it for the principal, so it would block this orchestrator outright. Here the orchestrator’s IAM policy carries kms:CreateGrant constrained three ways: by kms:GrantOperations to Decrypt, by kms:GranteePrincipal to the twelve known spoke roles, and by kms:GrantConstraintType to EncryptionContextSubset. The broadest grant it can create is one context-bound decrypt for one approved event-processor role.
Eventual consistency. Create, retire and revoke all propagate on the terms above. The GrantToken from CreateGrant closes the gap. Pass it on the cryptographic call (--grant-tokens on the CLI, GrantTokens in the SDK, up to ten per request). KMS then uses the token instead of waiting for the grant to be known everywhere.
Ending a grant: retire vs revoke
RetireGrant is the routine end. The grant itself decides who may call it: the retiring principal named in the grant, the grantee if the operations list includes RetireGrant, or the AWS account the grant was created in. It takes either the original GrantToken or the KeyId/GrantId pair, and the token is the only way to delete a grant that has not finished propagating. kms:RetireGrant in an IAM policy is how that account passes the ability to one of its principals. On its own the permission retires nothing, and it has no effect in a key policy or an RCP. An IAM Deny on it is the way to stop a principal retiring grants.
RevokeGrant is the administrative end, used to take permissions away rather than hand them back. It needs kms:RevokeGrant on the key, takes KeyId and GrantId, and will not accept a grant token.
Both delete the grant, and both appear in CloudTrail with the caller attached. A RetireGrant called by the grantee reads as a worker finishing its job; a RevokeGrant called by a key administrator reads as permission being pulled.
A grant that code creates for itself does not disappear when that code exits. A Lambda that grants to itself and then crashes leaves the grant standing until RetireGrant, RevokeGrant, or deletion of the key removes it. That is why the orchestrator sits outside the handler: the shared-services side creates the grant, hands the token to the Lambda, and deletes the grant on completion or after a bounded timeout.
The operations list and the encryption-context constraint are the prevention half of this control, evaluated by KMS on every call. The detection half is the CloudTrail pair: a CreateGrant with no matching RetireGrant is a permission nobody took back, and that query is the monitoring this design asks for.
Worked example
t = 0, the event arrives. EventBridge in spoke 222222222222 delivers a message to the payload-processor Lambda queue. The orchestrator in the shared-services account receives a companion control-plane event naming the target spoke role and the envelope metadata.
t + ~50ms. CreateGrant. The orchestrator calls:
aws kms create-grant \
--key-id arn:aws:kms:eu-west-1:999999999999:key/abcd1234-... \
--grantee-principal arn:aws:iam::222222222222:role/EventProcessorRole \
--retiring-principal arn:aws:iam::999999999999:role/GrantIssuer \
--operations Decrypt \
--constraints EncryptionContextSubset={envelope-id=evt-abc123,tenant=acme} \
--name evt-abc123
KMS evaluates the key policy and the orchestrator’s IAM policy, including the kms:GrantOperations and kms:GranteePrincipal conditions on it, creates the grant, and returns GrantId and GrantToken. A CreateGrant entry lands in the shared-services CloudTrail recording the key ARN, the grantee principal, the operations list, the constraints, and the grant id in responseElements. The grant token appears in no log entry. The orchestrator ships the token with the message.
t + ~150ms, the Lambda decrypts. The handler calls kms:Decrypt with GrantTokens=[token], the ciphertext, and the encryption context. The context has to be in the request, because that is what the EncryptionContextSubset constraint is matched against. The key policy does not name the spoke role; the token identifies the grant that does. KMS matches the constraint, accepts the token without waiting for propagation, and returns the plaintext data key. A Decrypt entry lands in the shared-services CloudTrail with the encryption context in requestParameters. A second copy lands in the spoke’s trail, because a cross-account KMS operation is logged in both the caller’s account and the key owner’s.
t + ~800ms, the handler exits. The Lambda emits a completion signal. The orchestrator calls kms:RetireGrant with the KeyId/GrantId pair. A RetireGrant entry lands. The deletion propagates on the same eventual-consistency terms as the creation, so the grantee can briefly still use the permission; once it is gone, reuse of the token returns AccessDeniedException.
Three entries in the shared-services trail for one event, with the Decrypt duplicated in the spoke’s. The grant existed for under a second. No policy was edited. The spoke role never held blanket decrypt permission on the key.
Failure modes worth naming. If the Lambda crashes before the orchestrator retires the grant, the grant stands until a timeout sweep calls RetireGrant. If the orchestrator is compromised, the key owner calls RevokeGrant on every outstanding grant and rotates credentials, and the blast radius is those in-flight grants. A leaked GrantToken is nonsecret by design and only usable by the grantee principal named in the grant, so the exposure is the spoke role’s credentials rather than the token. Skipping the token on the first call puts a sub-second handler in a race with propagation and returns AccessDeniedException. And the sweep is not optional: grants cap at 50,000 per key, so a fleet that creates faster than it retires eventually meets LimitExceededException on CreateGrant and stops decrypting anything.
One more asymmetry in the audit trail. A denied cross-account request is logged only in the caller’s account, so the token replay that returns AccessDeniedException shows up in the spoke’s trail and not in the shared-services trail where the key lives. An investigation run from the key owner’s account alone sees the successful decrypts and none of the rejected attempts, which is an argument for an organization trail rather than a per-account one.
What’s worth remembering
- An allow has three sources. Key policy, IAM policy or grant; any one of them allows, no allow anywhere denies, and an explicit deny wins.
- Grants fit per-execution permission. One key, one grantee, a fixed operations list, an optional encryption-context constraint, allow-only, no expiry.
- Pass the GrantToken immediately. Propagation is usually under five minutes, so the token from
CreateGrantis what makes the first call work. - CreateGrant rivals PutKeyPolicy. From a policy it can grant operations the caller lacks, so constrain
kms:GrantOperationsandkms:GranteePrincipal. - Three CloudTrail entries per execution.
CreateGrantandRetireGrantjoin on grant id; theDecryptbetween them records the encryption context. - 50,000 grants per key. Outstanding grants cap concurrent grant-authorised calls on one key, and the quota is adjustable.