Astra Qwen edit used 50 forward passes and Python, not raw-weight inspection alone

The honnibal.dev post enumerates eight experiments. Only experiments 1, 2 and 4 operated on raw floats without Python or network execution; all language-model results used code interpreters and repeated forward passes. Experiment 8, the headline result, ran the model on up to 50 batches of practice problems before editing 0.016% of weights.

Raw-weight inspection succeeded on 17K–172K parameter networks when the task was naming a training objective or correcting a single recorded failure. Scaling the same protocol to 800M-parameter models produced no measurable gain, as shown by the failed Gemma-2-9B trial listed as experiment 7a.

The separation of techniques indicates that current models can replicate human ML engineering workflows when given REPL access, yet lack a unified internal representation that would allow direct weight authoring at scale. This gap matches patterns observed in prior interpretability work where circuit-level edits remain limited to toy models.

Future claims will require explicit disclosure of tool use and parameter count per technique. Without such controls, aggregate success rates conflate distinct capabilities and overstate progress toward direct weight-level understanding.