Multi-Agent Reliability in AML/KYC Compliance Workflows
A pre-registered experiment measuring whether decomposing a compliance agent into a multi-agent system actually improves decision repeatability, across 41,400 scored runs and five open-weight model families.
Designed a controlled experiment that isolates architecture as the only variable: a single-agent and a multi-agent compliance platform built on the same model, rulebook, and tools, then measured against decision repeatability, label agreement, and cost.
Ran 18 sweeps totalling 41,400 scored runs across five open-weight model families on an externally authored AML triage benchmark.
Found that decomposition changes repeatability in a model-dependent way. Some models gained repeatability but lost accuracy under the multi-agent system, others showed the reverse, and some barely changed. Most re-run decision changes traced back to request-history cache state rather than sampling temperature, an effect the multi-agent system's chained calls amplified.
Built the RAG retrieval pipeline that grounds agent answers in the regulatory rulebook, using LangGraph, LangChain, and ChromaDB.