TRACER-7B records 11.4 conversion F1 gain on real customer-service cohorts

The paper introduces TRACER, a two-stage simulator. Supervised fine-tuning on real dialogues is followed by multi-turn reinforcement learning that combines hierarchical outcome- and trajectory-level rewards with deviation-aware advantage modulation. This addresses reward sparsity and credit assignment across extended interactions. On held-out real sessions the model also generalizes to out-of-distribution scenarios and passes human Turing tests at near-chance identification rates.

The Dynamic Marketing Benchmark built on TRACER shows that higher response quality scores do not translate into higher conversion rates. This decouples surface fluency from behavioral outcomes and exposes a measurable gap between standard LLM metrics and downstream task success. The result directly tests alignment between simulated intent evolution and observed user trajectories.

Prior simulators optimized token-level or single-turn plausibility and left multi-turn consistency unmeasured. TRACER’s trajectory-level rewards close that loop, producing cohorts whose aggregate conversion statistics match real distributions within tighter error bounds. Operational pipelines that rely on synthetic users for policy iteration now have a verifiable calibration target.

Next steps include integration into production A/B test platforms and extension to additional verticals where conversion or retention serves as the primary outcome signal.