Qwen3 8B models show largest Safety-suite deviation from no-memory baseline after stale recency injection
Persistent-memory agents over-trust stale facts in a capability-dependent manner. Larger Qwen3 and Llama models deviate most from tool baselines when recency cues are present. Mitigation efficacy also scales with model size.
The arXiv paper evaluates a frozen action-scored benchmark across Qwen3 0.6/1.7/4/8B checkpoints. Benefit-suite accuracy remains near 1.0 at every scale when only the stored fact solves the task. Safety-suite accuracy drops below the no-memory control once a recency cue is added, with the drop magnitude increasing with parameter count. Removing the memory label increases over-trust uniformly; source authority remains flat across sizes.
The pattern replicates on Llama-Instruct and on RGB and MisBench. Mitigation results are scale-dependent: metadata exposure restores accuracy only for the 4B and 8B models, while pre-resolving the conflict is required for the two smaller checkpoints. Position effects reverse sign between 1.7B and 8B.
Persistent-memory agents are already embedded in workflow tools that store user preferences and prior tool outputs. When those stores contain stale values that appear current, larger models override live tool responses at higher rates, raising the probability of incorrect actions in production pipelines. Smaller models fail differently, requiring different guardrails.
Deployment of 8B-class agents with persistent memory therefore requires capability-specific conflict resolution rather than uniform metadata tags. Cross-size contrast tests in the paper already show the interaction is statistically reliable.
Qwen3-8B agents: stale-recency over-trust rate exceeds 0.75 on Safety suite within production workflows by end of 2027
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2609.01852)
- [2]Supporting Source(https://qwenlm.github.io/blog/qwen3/)
- [3]Supporting Source(https://arxiv.org/abs/2502.XXX)