GLM 5.2 VAT benchmark records 7 pence net error on 59 transactions

The benchmark isolated GLM 5.2 on GCP with access only to a minimal bash harness, pre-authenticated accounting CLI, and internet for operational API calls. Human ground truth was derived from dual-verified 2026 Q1 Vineyard Finance books extracted via Claude Fable 5. The model ingested bank feed lines plus PDF receipts and optional user notes, then executed all ledger entries through the CLI without vision requirements.

Scoring applied six correctness criteria per transaction against the final software state. Net VAT position deviated by seven pence; all other boxes matched exactly. The model queried reverse-charge VAT rules specific to the SaaS provider but showed no evidence of external data leakage or test awareness gaming beyond one self-referential remark.

This result extends prior LLM accounting evaluations by demonstrating end-to-end CLI execution rather than isolated classification. Operational cost falls two orders of magnitude below the 750-2100 GBP quarterly fee range cited for SME outsourcing. The seven-pence residual error localizes to a single rounding interaction on a reverse-charge line.

Deployment timelines for similar agentic bookkeeping systems will be tested by whether error rates on equivalent 50-plus transaction sets remain below one pound through 2026.