How to Filter SNS Messages per Subscription

The situation

A product platform emits a platform-events SNS topic with roughly a million messages a day. Each message describes something that happened in the platform: an order placed, a user signed up, a billing cycle closed, a health probe result, a test-fixture execution. Events carry a consistent JSON body with fields like {"type": "OrderPlaced", "region": "eu-west-1", "priority": "high", "environment": "prod", "payload": {...}} and message attributes on the SNS side like type = "OrderPlaced", region = "eu-west-1", priority = "high".

Four services want to subscribe, each with a different slice:

1. Customer-service alerting, wants only high-priority events that affect customer-visible behaviour.
2. Region-specific analytics, one subscriber per region, each wanting events matching a single region code.
3. Security audit, wants all authentication-related events regardless of priority or region.
4. A catch-all archive, wants every event that isn’t from the test fixtures.

Before SNS filter policies, each subscriber was a separate topic with the producer duplicating messages. The platform wants to consolidate onto one topic and let each subscription decide what it wants.

What actually matters

Before reaching for the subscription editor, it’s worth asking what SNS actually supports.

SNS message filter policies are JSON documents attached to a subscription. When SNS receives a message on the topic, it evaluates each subscription’s filter against the message’s attributes (or, optionally, the message body) and delivers only to subscribers whose filter matches. The producer is unaware of the filters; subscribers own their own selectivity.

The first thing to ask is: what are we filtering on, attributes or body?

SNS supports two scopes:

• FilterPolicyScope: "MessageAttributes" (default). The policy matches against the message’s MessageAttributes map, a flat set of typed key-value pairs the publisher sets at publish time.
• FilterPolicyScope: "MessageBody". The policy matches against the message body itself, which must be JSON. Deep paths into the body are supported.

Both scopes use the same filter policy operators.

The second thing to ask is: what operators are available?

• Exact match: ["high"] matches attribute values of "high".
• Anything-but: [{"anything-but": ["test"]}] matches anything except the listed values.
• Prefix: [{"prefix": "AUTH_"}] matches values starting with the prefix.
• Suffix: [{"suffix": ".json"}].
• Numeric: [{"numeric": [">=", 100]}], with >, >=, <, <=, =, and range forms ["numeric", [">", 0, "<", 100]].
• Exists: [{"exists": true}] matches when the attribute is present.
• OR across values: ["eu-west-1", "eu-west-2"] matches either.
• AND across keys: {"type": ["OrderPlaced"], "region": ["eu-west-1"]} requires both.
• CIDR (for string attributes shaped like IPs): [{"cidr": "10.0.0.0/24"}].

The third thing to ask is: is the filter rich enough to do the job? For simple “deliver when this attribute equals this value,” attribute scoping is cheaper and faster. For “deliver when this nested field in the body is over a threshold,” body scoping is needed. Body scoping is slightly slower because SNS has to parse JSON and walk paths, but it’s still fast enough for typical workloads.

And finally: how many subscriptions can one topic support? The hard limit is 12.5 million subscriptions per topic, but at practical scale the number that matters is the soft quota on subscription filter policy complexity, each filter policy has limits on depth (5), nested arrays (100 values), and combinations (100 per key). Reaching those limits is rare but worth knowing before designing a filter that walks deep into the body.

Side by side

Reading the table by subscriber:

• Customer-service alerting, wants priority = "high" and type in a specific list. Attribute scope works; filter is {"priority": ["high"], "type": ["OrderFailed","PaymentDeclined","RefundIssued"]}.
• Region-specific analytics, one subscription per region, filter {"region": ["eu-west-1"]} etc. Attribute scope.
• Security audit, wants type matching a prefix AUTH_. Attribute scope with [{"prefix": "AUTH_"}].
• Catch-all archive, wants anything except events with environment = "test". Attribute scope with {"environment": [{"anything-but": ["test"]}]}.

None of the four need body scope; all four work with attribute scope. Body scope would come in if the filter were “priority based on a nested payload field.”

Constructing the filter policies

Customer-service alerting. Attribute policy:

{
  "priority": ["high"],
  "type": ["OrderFailed", "PaymentDeclined", "RefundIssued", "AccountLocked"]
}

Filters combine with AND across keys and OR within an array. This policy matches messages with priority = "high" AND (type = OrderFailed OR PaymentDeclined OR RefundIssued OR AccountLocked).

Region-specific analytics. Four subscriptions, one per region:

{ "region": ["eu-west-1"] }
{ "region": ["eu-west-2"] }
{ "region": ["us-east-1"] }
{ "region": ["ap-southeast-1"] }

Or one subscription with a multi-value filter:

{ "region": ["eu-west-1", "eu-west-2"] }

The choice depends on whether each region-specific pipeline is distinct (separate SQS queues per region) or one pipeline consumes all. If each region needs its own processing, four subscriptions; if one pipeline routes internally, one subscription.

Security audit. Prefix match:

{ "type": [{"prefix": "AUTH_"}] }

Every event whose type starts with AUTH_ is delivered. The producer keeps the convention: AUTH_LoginSuccess, AUTH_LoginFailed, AUTH_PasswordReset. New auth events automatically route to the audit subscriber without subscription changes.

Catch-all archive. Anything-but:

{ "environment": [{"anything-but": ["test"]}] }

Every event that’s not tagged as a test fixture. Gives the archive a clean stream for compliance without fixture noise.

The flow, drawn

Attribute scope vs body scope

Most filters can be expressed in either scope. Attribute scope is the default because:

1. Producers typically set attributes for things subscribers care about routing on (priority, region, type).
2. Evaluation is faster because SNS doesn’t have to parse JSON.
3. Delivery cost is the same, the bill is per matched delivery, not per evaluation.

Body scope is the correct answer when:

• The routing field is genuinely nested in the payload and adding a duplicate attribute would be noise.
• The subscriber needs to filter on a computed property of the message body (payload.amount > 1000).
• The team can’t change the producer to add new attributes but owns the subscribers.

Body scope example:

{
  "payload": {
    "amount": [{ "numeric": [">", 1000] }]
  }
}

Matches messages whose JSON body has payload.amount > 1000. Requires FilterPolicyScope: "MessageBody" on the subscription.

Pitfalls worth naming

Type coercion. Filter policies are strict about types. Numeric operators work only when the attribute is sent with the DataType: "Number" (for attribute scope) or the body field is a JSON number. An attribute sent as DataType: "String" with value "42" does not match [{"numeric": [">", 0]}]. Publishers must set types correctly.

Missing attributes. A filter requiring {"priority": ["high"]} on a message without a priority attribute doesn’t match. Use [{"exists": true}] or [{"exists": false}] to explicitly handle presence/absence.

Policy syntax errors. A subscription with a malformed filter policy silently receives no messages. aws sns get-subscription-attributes shows the configured policy; a dry-run test against a known message body helps catch mistakes.

Unfiltered delivery. A subscription with no filter policy receives every message. When splitting a monolithic subscriber into multiple filtered ones, explicitly set a filter on each or one will be the implicit catch-all.

Hierarchy of filters. Filter policies apply to subscriptions, not to the topic. Each subscription has its own. Moving a filter from “on the producer” to “on each subscriber” concentrates filter logic where it’s easiest to change.

Cost shape

SNS pricing is per published message and per delivery. With filter policies:

• Every published message is charged once at the publish rate.
• Every delivered message is charged at the delivery rate (different per protocol. SQS, Lambda, HTTP, etc.).
• Filtered-out messages are not delivered, so they don’t incur delivery charges.

For a topic at a million messages a day with four subscribers each receiving on average 250,000 of them, the delivery bill is roughly the same as having four separate topics with producer-side filtering. The benefit is operational: one topic, one resource policy, one CloudWatch dashboard. Filter policies pay back in maintenance, not in direct cost.

What’s worth remembering

1. Filter policies live on subscriptions. Each subscription has its own filter; SNS evaluates before delivery; non-matching subscribers see nothing.
2. Attribute scope is the default. Attributes must be typed (String, Number, String.Array). Faster, simpler; works for most routing fields.
3. Body scope lets filters walk into JSON payloads. FilterPolicyScope: "MessageBody"; required when the routing field is nested.
4. Operators cover the common cases. Exact, prefix, suffix, anything-but, numeric comparisons, exists, CIDR. OR within an array, AND across keys.
5. Missing attributes don’t match positive filters. Use exists: true/false when presence itself is the condition.
6. Type correctness matters. Numeric filters need numeric types, not string-formatted numbers.
7. No filter = everything. Catch-all subscribers simply have no filter policy; explicit is better than implicit.
8. Filtering doesn’t add cost. Same publish + delivery pricing as without filters; the operational savings come from fewer topics.
9. Filter-policy complexity has limits. Depth 5, combinations 100 per key; worth checking when policies grow expressive.
10. Changes are subscription-level. Adding a filter, changing a value, introducing a new subscriber, all are set-subscription-attributes calls; the topic and producer are unaffected.

One topic, many subscribers, one filter policy per subscriber. The producer emits every event; subscribers decide what they want; the platform team maintains four subscription attributes rather than four topics. The work isn’t picking which subscribers to support, it’s letting each one express its selection criteria in a filter policy and letting SNS handle the rest.