TEAM-Design Allocates Replays Across 6 Clinical Settings Where No Human-AI Workflow Beats Both Baselines

The paper introduces TEAM-Design, a sampling rule that computes two replay probabilities per task. Probability rises when the missing baseline outcome is difficult to predict from observed data and when the comparison is closer to failing the superiority threshold. It falls when replay cost is high. The rule is proven to solve the budgeted design problem exactly and to maintain error-rate control even under random draw from the computed probabilities.

Reanalysis of six chest X-ray and radiology studies showed every human-AI workflow failed to beat both human-alone and model-alone performance. A coding benchmark yielded one workflow that succeeded. Synthetic and semi-synthetic experiments confirmed TEAM-Design outperforms variance-based allocation when one comparison is markedly harder than the other, yet underperforms when difficulties are balanced.

Existing agent benchmarks and Bayesian methods optimize parameter estimation rather than the binary keep-or-replace decision. By forcing explicit measurement of both counterfactuals, TEAM-Design surfaces cases where collaboration increases error rates or masks accountability, directly addressing ethical concerns over diffused responsibility in clinical and engineering teams.

Next deployments will require pre-specified replay budgets and documented probability vectors before any human-AI workflow receives production approval. Regulators can treat the resulting superiority claims as verifiable only when both baselines have been replayed according to the design.