Architecting a High-Throughput ISO 20022 Payments Switch

A payments switch sits between originators — mobile apps, merchant acquirers, core banking systems — and a set of downstream rails: card networks, real-time gross settlement systems, ACH-equivalents, and increasingly, ISO 20022-native schemes like SEPA Instant or the various RTP networks. Its job sounds simple: take a payment instruction, validate it, route it to the correct rail, and report back what happened. In practice it is one of the least forgiving pieces of infrastructure you can build, because every message represents money that must move exactly once, in the right amount, to the right party, even when a downstream partner is timing out, a network partition splits your data centers, or a duplicate message arrives because a client retried after a socket reset.

ISO 20022 changes the shape of the problem more than it changes the difficulty. The standard replaces older fixed-format and proprietary XML dialects with a structured, XML- or JSON-encoded message set (pain, pacs, camt families) that carries much richer remittance and party data. That richness is valuable for reconciliation and screening, but it also means a switch has to validate deeply nested, semantically constrained documents — not just check that a field is present, but that a debtor agent BIC is structurally valid, that a currency code matches an ISO 4217 entry, that a structured remittance reference respects the scheme's own usage guidelines, which frequently diverge from the base standard. Any switch design has to treat ISO 20022 as a family of scheme-specific dialects, not a single wire format.

The constraints that actually drive the architecture are: sustained throughput with tail-latency guarantees under partner backpressure, strict exactly-once delivery semantics per instruction, multi-rail heterogeneity (each rail has its own timeout behavior, retry semantics, and settlement finality model), and an audit trail that a regulator or a dispute team can reconstruct months later without ambiguity. None of these are solved by picking a fast message bus; they're solved by the data model and the state machine underneath it.

The central abstraction is not a 'transaction' row that gets mutated in place — it's a payment instruction represented as an append-only sequence of domain events: Received, Validated, RoutingDecided, SubmittedToRail, RailAcknowledged, Settled, Rejected, Reversed. Each event is immutable, timestamped, and carries the actor that produced it. The current state of a payment is a projection over its event stream, not a mutable status column. This matters specifically because payment lifecycles are not linear — a real-time payment can be submitted, time out from the switch's perspective, and then have the rail confirm it minutes later; an event-sourced model lets you reconcile that late confirmation against the correct instruction without a special-case status field for 'unknown, pending investigation.'

Every instruction is keyed by a natural idempotency key derived from the originator's own identifiers — typically a combination of originating system ID and client-supplied end-to-end reference — never by a switch-generated UUID alone, because the switch cannot be the source of truth for whether a request is a retry or a new instruction. On ingest, the first thing the switch does is an idempotency check against this key with an atomic insert-if-absent; if the key already exists, the switch returns the previously computed outcome rather than re-processing. This single mechanism removes an entire class of double-execution bugs that would otherwise require careful client-side retry discipline the switch cannot enforce.

ISO 20022 messages are parsed into an internal canonical representation before anything else touches them. Scheme-specific adapters translate a pacs.008 or a pain.001 into this canonical form and back; the routing and validation core never sees raw XML. This adapter boundary is what lets the switch add a new rail or a new scheme version without touching the core state machine — a discipline that pays for itself the first time a scheme publishes a breaking usage-guideline update mid-year.

On the write path, an inbound instruction hits a thin ingestion service that does three things synchronously: schema validation against the relevant ISO 20022 message definition, idempotency-key resolution, and a first-pass sanctions/format screen. If all three pass, the instruction is persisted as a Received event in the append-only store and a decision is handed back to the caller only once that write is durably committed — not before. This is a deliberate choice: the caller's synchronous acknowledgment reflects durability, not final settlement, and the API contract has to make that distinction explicit rather than implying 'success' means 'money moved.'

Routing itself happens asynchronously, driven by a message bus (Kafka or an equivalent durable log) rather than direct service calls. A routing decision consumes the Received event, applies rail-selection logic — based on currency, amount thresholds, scheme reachability, and current rail health — and emits a RoutingDecided event, which in turn triggers a rail-specific adapter to submit the translated message. Decoupling ingestion from rail submission through a durable log means a slow or unavailable rail backs up its own consumer group without blocking ingestion of unrelated payments, and it gives you a natural replay mechanism: if a rail adapter crashes mid-submission, the next instance picks up from the last committed offset rather than needing bespoke recovery logic.

The read path — status queries, reconciliation reports, dispute lookups — is served from materialized projections built off the same event stream, kept eventually consistent with a bounded lag that's monitored explicitly. Reconciliation against rail-provided settlement files is a separate, batch-oriented consumer of the same event log, matching camt.053/054-style statement entries back to instructions by end-to-end reference and emitting Settled or Discrepancy events. Keeping reconciliation as just another consumer of the canonical event stream, rather than a bolted-on nightly job against a mutable ledger table, keeps the audit trail internally consistent.

The hardest failure mode in a payments switch is not the rail being down — that's observable and recoverable — it's the ambiguous outcome, where the switch submits an instruction to a rail and then loses the ability to determine whether it was accepted before a timeout or a connection reset occurred. The design response is to never treat 'no response' as 'failure.' A submission that times out moves the instruction into an explicit InFlightUnknown state rather than Rejected, and no compensating action (retry, reversal, customer notification) is taken until either the rail's own status-query API resolves the ambiguity or a reconciliation file confirms one way or the other. Automatically retrying an unacknowledged submission without this check is how duplicate payments happen in production.

Circuit breakers wrap every outbound rail adapter, tuned per rail based on that rail's own latency and error profile rather than a single global threshold, because a card network and a real-time rail have entirely different normal operating envelopes. When a breaker opens, instructions destined for that rail are held in the durable log rather than dropped or rerouted silently — routing changes are a business decision (does this amount qualify for an alternate rail under the scheme rules) and should never be an implicit side effect of infrastructure health.

Correctness at the data layer relies on the append-only event log being the single source of truth, with all derived views — balances, statuses, reports — recomputable from it. This is expensive to maintain compared to a simpler mutable-row design, but it is the only approach that survives an auditor asking 'show me exactly what this system believed at 14:32 on the day of the dispute,' because the event log answers that question by construction rather than by trusting that nobody overwrote history.

Horizontal scale in this architecture comes from partitioning the event log by instruction key, so that all events for a given payment land on the same partition and are processed in order by a single consumer, while unrelated payments scale out across partitions independently. This gives strict per-instruction ordering without a global lock, at the cost of needing careful partition-key design up front — repartitioning an event-sourced system after the fact is a genuinely hard migration, not a config change, so this is a decision made once and defended.

Operability depends heavily on making the InFlightUnknown and Discrepancy states first-class, queryable, and alertable, because a switch that hides ambiguity behind a generic 'processing' status is one an operations team cannot actually run. Dashboards should surface rail-by-rail breaker state, per-scheme validation rejection rates (which spike visibly after a scheme's periodic usage-guideline update), and the age distribution of in-flight instructions, since a growing tail of old in-flight payments is usually the earliest signal of a downstream reconciliation gap.

The trade-offs are real and worth naming plainly. Event sourcing adds operational complexity — schema evolution for events is a discipline unto itself, and teams new to the pattern underestimate the cost of projection rebuilds at scale. We would not choose it for a low-volume internal payments flow where a conventional transactional table with careful locking is simpler to reason about and cheaper to build. We would also not attempt to build scheme-specific ISO 20022 validation from scratch in-house long-term; vendored or open validation libraries tied to the scheme's published schema versions reduce the maintenance burden of tracking usage-guideline churn. The switch pattern earns its complexity specifically at the point where multi-rail heterogeneity, strict idempotency requirements, and audit obligations converge — which is precisely the environment most production payments infrastructure lives in.