h_add heuristic records 0.931 mean Spearman correlation predicting LLM planning failure on held-out STRIPS domains

The paper isolates structural from semantic factors by renaming predicates while preserving state transitions and optimal plans. LLM success falls from 67.5 percent to 40 percent at full renaming with zero refusals. This drop occurs uniformly across domains, indicating that predicate vocabulary alignment with pretraining data drives performance more than plan length or object count. ARC supplied with the same features underperforms a linear baseline, confirming that episodic retrieval adds no structural signal beyond what simple counts already capture.

Established delete-relaxation heuristics already separate structural hardness from LLM-specific semantic sensitivity. Prior work on PDDL generalization, including Valmeekam et al. 2023 on plan length scaling and Huang et al. 2024 on predicate abstraction, showed similar vocabulary dependence but lacked controlled renaming protocols. The current results tighten that link by holding optimal plans fixed. Hybrid routing therefore reduces to per-LLM calibration curves rather than improved structural estimators.

Operationally, systems should route instances to symbolic solvers when h_add exceeds domain-specific thresholds derived from the target LLM's observed failure rate. This approach requires logging predicate-token overlap statistics per model rather than further heuristic refinement. Future benchmarks must report both structural scores and semantic renaming curves to avoid conflating the two failure modes.