MARQ raises median F1 to 0.80 on Claude Opus 5 versus 0.55 baseline on NASA requirements

MARQ is a modular prototype that supplements LLMs with criterion-specific knowledge stacks operationalizing INCOSE requirement quality criteria. It was tested on an expert-labeled NASA requirement set across 100 executions per model using Claude Opus 5, GPT-5.6 Sol, and Gemma 3 12B. The design isolates the effect of SE knowledge infusion from raw model capability.

Claude Opus 5 improved both defect detection and false-alarm control. GPT-5.6 Sol gained detection power and execution consistency yet produced more false alarms. Gemma 3 12B reached parity with prior proprietary baselines while remaining highly variable. Gains concentrated in Correctness and Conformance criteria, with documented trade-offs between misses and false positives within individual criteria.

Requirement defects propagate through AI-driven systems development, directly limiting adoption where compliance and safety standards apply. Model-dependent response patterns indicate that uniform prompting fails to address criterion-level variance. Human-in-the-loop review remains necessary where false-alarm rates rise.

Criterion-specific interventions and repeated execution protocols will be required to stabilize performance before production deployment in regulated environments.