arXiv:2610.00015 defines Praxa harness with four evidence lanes and zero superiority claims
Praxa supplies an evidence-bound state machine for agent execution but demonstrates no superiority on benchmarks or production tasks. All four reported lanes meet coverage floors yet record equal or higher token costs without outcome gains. Future work requires controlled trials with falsifiable production metrics.
Praxa separates proposal, authority, dispatch, read-back, reconciliation, and promotion into explicit deterministic stages. The architecture instruments 363 source files and enforces admission plus external verification before any side effect. Four evidence lanes document the implementation: repository audit, Terminal-Bench Core 0.1.1 pilot, post-debug coordination-proxy comparison, and deployed configuration traces.
Terminal-Bench results show identical 17/36 strict passes for baseline and reliability layer, with the latter consuming 37.49% more input tokens and 50.73% more output tokens. The coordination-proxy trial completed 180/180 runs for both arms with equal accuracy and zero protected violations, while the candidate used 37.11% fewer tokens and 11.63% fewer steps. No lane reports adversarial security results or production metrics.
The paper's own abstract states that current evidence does not establish adversarial security, production safety, or user benefit. Related agent harness work such as LangChain's LangGraph checkpoints and Anthropic's tool-use evaluation suites similarly publish pass rates without controlled outcome deltas. Praxa therefore supplies a testable state machine rather than a performance advance.
Next deployments must publish pre-registered outcome measures on at least 500 live tasks with independent reproduction to move beyond architecture documentation.
Praxa: No production deployment will publish >5% outcome lift versus baseline on 500+ live tasks within 18 months
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2610.00015)
- [2]Supporting Source(https://arxiv.org/pdf/2610.00015)