ArXiv 2610.14115 finds majority of agentic DS pipelines embed data leakage
ArXiv 2610.14115 demonstrates that agentic data science evaluations centered on accuracy alone conceal data leakage and missing domain knowledge. The resulting over-optimistic scores correlate with failure modes that later translate into biased deployed models. Principled failure-mode testing must become standard to prevent systemic unfairness.
The paper evaluates three agentic frameworks on 220 tasks plus a controlled test suite targeting six failure modes. It records a positive correlation between data leakage incidence and reported accuracy, showing that standard leaderboards reward the very errors they claim to measure. Qualitative review of Kaggle submissions further reveals agents lack domain-knowledge adjustments that human winners apply to prevent leakage.
Data leakage produces inflated metrics that mask poor generalization. When such pipelines reach production they embed systematic performance gaps across subgroups, directly violating fairness requirements in regulated domains. Current benchmark design, focused solely on predictive score, therefore incentivizes the very shortcuts that later produce disparate impact.
Existing agent harnesses inherit the same evaluation blind spots documented in prior leakage studies. Extending test suites with leakage probes and domain-knowledge checks is required before agentic systems can be trusted for high-stakes modeling.
Next steps include integration of the released test suite into standard agent leaderboards within six months and mandatory leakage audits for any deployment claiming regulatory compliance.
Agentic DS test suites: at least two major benchmarks will add mandatory leakage probes by Q3 2027
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2610.14115)
- [2]Supporting Source(https://proceedings.mlr.press/v139/kaufman21a.html)
- [3]Supporting Source(https://arxiv.org/abs/2305.16562)