Designing a KYC/AML Screening and Case-Management Pipeline

A KYC/AML screening and case-management system has one job that sounds simple and isn't: decide whether a person or entity is who they claim to be, and whether continuing to do business with them creates legal exposure. Every new customer, counterparty, or beneficial owner needs to be checked against sanctions lists, politically-exposed-person (PEP) registries, and adverse-media sources, then re-checked whenever those lists change or the subject's risk profile shifts.

The engineering tension sits between two failure modes that pull in opposite directions. Missing a true match is a legal and regulatory failure — the kind that shows up in enforcement actions, not bug trackers. Over-matching floods analysts with false positives, and a backlog of unreviewed cases is itself a compliance finding. A pipeline that is either too permissive or too noisy has failed at its actual job, regardless of how well it scales.

There's also a temporal dimension most people underestimate: reference lists are not static. Sanctions lists update daily, sometimes intraday during geopolitical events. A subject who screened clean on Monday can be a confirmed match by Wednesday without ever touching your system again. The architecture has to treat 'screen once at onboarding' as necessary but not sufficient — ongoing rescreening is a first-class workflow, not an afterthought bolted on later.

Finally, everything has to be explainable after the fact. A regulator or auditor will ask, for a specific subject, on a specific date: what list version did you screen against, what was the match score, who reviewed it, and why was it cleared or escalated. If that trail isn't reconstructable from the data model itself, no amount of runtime correctness saves you.

The model separates three concerns that are easy to conflate: the subject being screened, the screening events run against them, and the case that aggregates findings needing human judgment. A Subject holds identity attributes (name, date of birth, national ID, jurisdiction) and a version number that increments on any material change. A Screening Event is immutable and pinned to a specific reference-list version and timestamp — it records 'we screened subject X, version N, against sanctions list version M, at time T,' full stop. Nothing about that record is ever edited after the fact; corrections happen by writing a new event, never by mutating history.

Match candidates produced by a screening event are their own append-only records, each carrying a match score, the matching algorithm and parameters used, and an initial disposition (auto-cleared, escalated). A Case is a separate aggregate that groups one or more matches requiring review, with its own state machine — OPEN, IN_REVIEW, ESCALATED, CLOSED_CLEAR, CLOSED_CONFIRMED — and every transition is an event with an actor, a timestamp, and a reason.

The reason to event-source the case rather than store it as a mutable row with a status column is almost entirely regulatory: the audit trail has to be the source of truth, not a side effect of logging. A mutable table can drift from what actually happened if someone patches a row directly; an event log by construction cannot. The cost is that read models — the case queue an analyst sees, the dashboard a compliance officer sees — become projections that need to be built and kept in sync, which is real complexity you take on deliberately, not by default.

Onboarding or a material subject update publishes a domain event onto a stream. Screening workers consume it and run the subject against sanctions, PEP, and adverse-media reference sets using a two-layer matching approach: deterministic matching on strong identifiers (passport number, national ID, date of birth) where available, combined with fuzzy name matching — typically token-based similarity plus phonetic algorithms — for the cases where identifiers are missing or the only signal is a name. Fuzzy-only matching at scale is a false-positive generator; anchoring it with deterministic identifiers whenever possible is what keeps the case queue survivable.

Scores from matching feed into an overall risk score that also weighs contextual factors — jurisdiction risk, product type, expected transaction profile. Clear non-matches auto-clear; matches above a low-confidence floor and below a high-confidence ceiling route to a case for human review; matches above the ceiling on a sanctions list typically block the relationship pending review rather than merely flagging it, since the downside of an active sanctioned relationship is categorically worse than onboarding friction.

Rescreening on list updates is where naive designs fall over. Re-running every subject against every list on every update doesn't scale and isn't necessary — most subjects couldn't possibly be affected by a given list delta. Instead, diff the list for additions and removals, then run a targeted match against only those deltas using an indexed lookup (a search engine tuned for fuzzy text matching, not a full table scan) to find which existing subjects are newly at risk. This bounds the cost of rescreening to the size of the list change rather than the size of the customer base, at the price of needing a reliable, well-tested diffing and indexing layer as a first-class component, not a cron-job afterthought.

Cases are queued to analysts by risk tier and current workload, with SLA timers that trigger escalation if a case sits unreviewed past a threshold appropriate to its severity. High-risk case closures — anything touching a sanctions match or a confirmed PEP — require dual authorization: one analyst's disposition is not sufficient to close the case, a second reviewer with distinct permissions has to concur. This maker-checker pattern is standard in regulated workflows precisely because a single point of human judgment is also a single point of failure.

Every analyst action — a note, a disposition, an escalation, a request for more documentation — is captured as its own immutable audit entry tied to the case's event log, with the analyst's identity and timestamp. This isn't just compliance box-checking; it's what makes the system debuggable when a decision is questioned months later.

Resolved cases are a valuable feedback signal for tuning match thresholds and risk scoring over time — a cluster of cases consistently auto-cleared by analysts at a given score band suggests the threshold is too conservative. That said, this pipeline deliberately keeps a human decision-maker in the loop for anything above the auto-clear floor; replacing analyst judgment with a fully automated scoring model raises an explainability problem regulators are not currently willing to accept, and it's not a trade-off worth making even where it's technically feasible.

The core failure-handling principle is fail closed, not fail open. If the screening service is unavailable during onboarding, the correct behavior is to block activation until screening succeeds — not to let the customer through and screen asynchronously later. An onboarding gate that can be silently bypassed by a downstream outage is a control that doesn't actually control anything.

Idempotency matters throughout: a subject-version plus list-version key on each screening event means a redelivered or retried message produces the same result rather than a duplicate case. Event publication uses an outbox pattern — write the domain event to the database in the same transaction as the state change, then relay it to the stream — so a crash between the write and the publish can't silently drop a screening trigger.

When an upstream reference-list source itself is unavailable, the system falls back to the last known-good list version rather than blocking indefinitely, but it flags every screening event run against a stale list as such, and a reconciliation job re-screens those subjects once a fresh list is available. Silent staleness is worse than visible staleness. Raw match evidence and the PII backing it are encrypted at rest and access-scoped separately from the case metadata itself, since the two have very different audiences — an analyst needs the case, not necessarily the raw underlying document.

Event sourcing the case aggregate buys an audit trail that's structurally impossible to falsify after the fact, at the cost of needing CQRS-style projections for every read pattern an analyst or dashboard actually needs — that's meaningfully more moving parts than a mutable status column, and it's only worth it because the audit requirement is non-negotiable here.

Targeted rescreening on list deltas, rather than full periodic rescans, trades a small bounded window of staleness — closed by a scheduled reconciliation pass — for a rescreening cost that scales with list churn instead of customer count. For a reference-list update that touches a few hundred entries, that's a very different operational profile than reprocessing a full customer base on every update.

We would not build a case-closure path that lets an ML model auto-close high-risk matches without a human in the loop, even where the model's precision looks strong in testing — the explainability and accountability requirements of this domain make that a control weakness, not an efficiency gain. And we would not let onboarding proceed on a best-effort, screen-later basis when the screening service is degraded; the latency cost of a synchronous, fail-closed gate is the price of the guarantee that nothing gets onboarded unscreened.