TORSI lifts Jev from 13.51 to 24.19 on RealtimeBench with 180 live-clock tasks
TORSI separates slow teacher analysis from fast student execution to close the gap between paused and live-clock performance. Primary data show a 10.68-point absolute gain on RealtimeBench that generalizes across models. The approach supplies a concrete mechanism for real-time agent viability without added inference cost.
RealtimeBench evaluates agents on 180 tasks where environments advance during LLM inference and tool calls. GPT-6.1-sol led paused evaluations yet trailed the faster Jev baseline once clocks remained live. TORSI decouples timescales: a slow teacher inspects failures in paused replays, generates reusable instructions, and validates them offline before the fast student applies them without additional calls.
Validation on held-out runs selected only instructions that raised scores. Two TORSI rounds produced the 10.68-point gain and transferred to three unrelated models without parameter updates. The method records no increase in per-step latency during live play.
Operational impact centers on separating deliberation from execution. Agents that previously required external oversight now carry pre-validated instruction sets that maintain effectiveness as state changes continue. Deployment records for similar offline-to-online loops in control systems show comparable retention of gains when validation thresholds exceed 0.15 progress points.
Next evaluation cycles will measure instruction stability over 50+ sequential games and test whether further rounds exceed 30.0 average without teacher recomputation.
Jev: average RealtimeBench score exceeds 30.0 within three additional TORSI rounds on the same 180-task set
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2610.20665)
- [2]Supporting Source(https://arxiv.org/abs/2503.14172)