Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns
Benchmark shows synthetic generators degrade fraud behavioral signals 17x-99x, challenging privacy AI assumptions.
A new arXiv paper introduces behavioral fidelity to expose how synthetic tabular generators fail to preserve temporal, velocity, and multi-account fraud signals essential for operational detection systems (Sajja et al., arXiv:2604.13125).
The benchmark defines P1-P4 taxonomy covering inter-event timing, burst structure, multi-account graph motifs, and velocity-rule triggers with a degradation ratio metric calibrated to real-data noise floor; row-independent generators CTGAN, TVAE, and GaussianCopula prove structurally incapable of reproducing P3 motifs (Proposition 1) or positive within-entity IET autocorrelation (Proposition 2), yielding 24.4x-39.0x composite degradation on IEEE-CIS Fraud Detection and 81.6x-99.7x on Amazon Fraud Dataset while TabularARGN reaches 17.2x (Sajja et al., arXiv:2604.13125; Xu et al., arXiv:1907.00503).
Prior statistical fidelity and AUROC-focused evaluations missed these sequential behavioral failures that real fraud systems rely on; related differential privacy work similarly overlooked entity-level temporal patterns now shown to extend to healthcare and network security domains, exposing gaps in assumptions about synthetic data for high-stakes financial AI (Abadi et al., arXiv:1607.00133).
AXIOM: Synthetic tabular generators erase temporal bursts and multi-account motifs that define fraud, rendering them unreliable for training financial security models despite passing traditional statistical checks.
Sources (3)
- [1]Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals(https://arxiv.org/abs/2604.13125)
- [2]Modeling Tabular Data using Conditional GAN(https://arxiv.org/abs/1907.00503)
- [3]Deep Learning with Differential Privacy(https://arxiv.org/abs/1607.00133)