KWBench contains 223 practitioner-sourced tasks from acquisitions, contract negotiations, clinical pharmacy, organizational politics, fraud analysis and incentive design. Each encodes one of six game-theoretic patterns: principal-agent conflict, signaling, mechanism design failure, strategic omission, coalitional dynamics or strategic interdependence. Models receive only raw inputs and a neutral task prompt with no problem-type cue; scoring requires a conjunctive check that the model avoided all predicted wrong paths before quality is assessed (arXiv:2604.15760).

Best model passed 27.9% of tasks. Top-two models agreed on only 31.7% of passes; 44 tasks were solved by exactly one of the top eight models. Routing across those eight models covered 50.7% of the benchmark. Conditional on passing, quality scores converged near 83% across models, yet unconditional scores diverged sharply. The same models that failed unprompted could name the correct game-theoretic concept when directly queried (arXiv:2604.15760). These results align with AgentBench findings that LLMs show capability drops when moving from prompted to open-ended autonomous regimes (arXiv:2308.14901).

Mainstream benchmarks such as GPQA have measured prompted expert-level reasoning yet omitted the preceding recognition step required in professional environments (arXiv:2311.12022). KWBench's mandatory failure-mode gating and raw-input format therefore surface a capability gap prior coverage treated as solved once execution accuracy rose. The benchmark indicates that ensemble routing or specialized recognizers may be required before LLMs can operate safely without constant framing in knowledge-work domains.