Text2Dashboard arXiv:2610.06914 reports 6/8 strict success on DataBrain metadata and SQL tasks
arXiv:2610.06914 presents a governed agent pipeline that converts natural-language requests into inspectable dashboards over enterprise data. Strict success reached 6/8 on core tasks and 100 percent on controlled fault scenarios, with model inference accounting for over 97 percent of observed runtime. The work isolates model proposals from deterministic control layers but remains limited to a single DataBrain deployment.
The system combines a Codex plugin with an Agent Runtime that enforces schema-constrained decisions, typed tools, read-only SQL, and deterministic Hooks for approval, audit, and recovery. Entity resolution, metadata discovery, dashboard composition, and multi-stage checks run under persistent state control while the model only proposes actions. All ten injected fault scenarios produced specified outcomes with no unapproved external effects.
Evaluation on eight real tasks showed metadata selection at 4/4 and SQL semantic correctness at 4/4, yet only 2/4 met the exact output-column contract. The final release passed all four single-panel dashboards, one two-panel task, and one refinement case; a parameterised task hit the step limit. These DataBrain-specific numbers do not claim general text-to-SQL performance.
The architecture separates model proposals from deterministic execution, addressing audit and failure-handling gaps that appear in unconstrained agent frameworks. Runtime data indicate that inference dominates cost, shifting optimisation focus from tool latency to model efficiency. Production use will require measurable gains in exact column fidelity and step-limit handling beyond the reported 6/8 baseline.
Next milestones include expanded task sets and direct comparison against manual dashboard construction times on the same frozen workloads.
Text2Dashboard: exact column-contract success on new DataBrain tasks will remain below 70 percent through December 2026 without additional schema fine-tuning.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2610.06914)
- [2]Supporting Source(https://arxiv.org/abs/2305.10645)
- [3]Supporting Source(https://arxiv.org/abs/2402.01030)