EvoSyn arXiv:2610.12593 lifts agent task success 13.3 points via learner-feedback synthesis revision
EvoSyn demonstrates closed-loop data synthesis by evolving the explicit generation space from learner feedback. Experiments record 13.3 point gains on two agent benchmarks while surfacing controllable factors missed by static synthesis methods. The approach supplies a concrete mechanism for turning policy weaknesses into targeted synthetic trajectories.
The framework maintains a revisable state of controllable interaction factors. A Designer agent maps observed weaknesses to new synthesis directions. A Planner converts those directions into executable trajectories. Feedback closes the loop by expanding the generation space when learner errors reveal previously implicit factors. On τ²-Bench and CAR-Bench the method produced both higher structural diversity and measurable coverage of previously under-sampled regions.
Data from the October 2026 submission record 13.3 point gains against the strongest synthesis baseline that already incorporated environment feedback and validation. Ablations show within-space adaptation accounts for roughly half the lift while space expansion supplies the remainder. The paper reports no external human annotations; all revisions derive from agent-internal success signals.
Closed-loop synthesis of this form directly supplies the missing mapping from observed policy gaps to concrete data-generation edits. Prior pipelines required manual redesign of prompts or environments after each evaluation round. EvoSyn automates that step, turning failure traces into updated factor lists that subsequent synthesis rounds must target.
Next deployments will test whether the same revision mechanism scales to multi-turn web agents and embodied simulators where factor spaces are larger and verification costs higher.
EvoSyn: Follow-up work will report >75% average success on CAR-Bench within 9 months when synthesis revision is applied to models above 70B parameters.
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2610.12593)
- [2]Supporting Source(https://arxiv.org/abs/2406.10205)