Weight-space archaeology of a compressed-GRPO collapse

Run qwen3-4b-4k-commeff-500 (Qwen3-4B-Base, MATH, PRF exact-k p=0.95, anchor + signed-EMA), collapsed at step ~152. Evidence: the step-100 and step-200 FSDP checkpoints with full AdamW state, the base model, WandB logs, and offline surgery on one RTX 5090. All numbers reproduce from research/scripts/weight_archaeology/. Full experiment log: LEDGER.md alongside this file.

What happened. The signed-EMA gradient correction rewrites every coordinate's gradient sign to match the anchor EMA M, and that sign pattern is frozen for the 20 ticks between anchor fires. AdamW then normalizes away the masked gradient's magnitudes (the clip at 1.0 was binding on every one of the 200 steps). Net effect: all 4 billion coordinates marched at a uniform ~0.4 lr per step in 20-tick coherent kicks along sign-of-dense-gradient directions. That walk carries no per-coordinate information but is loaded on functionally sensitive directions, so it steadily flattened the policy. Sharpening from learning outpaced the flattening until step ~76, then lost. The flattened policy leaks past its stop points at sampling temperature 1.0, lengths grow, truncation poisons the advantages, and the run unravels at ~150. The trainer's own metrics were structurally blind to all of it (masked-vs-masked ratios cancel), but the sampler's own token logprobs saw it 40 steps early.

1. The timeline, and the signal that was watching

Score collapses after 150. The fraction of rollouts hitting the 3072 cap breaks at 148 to 151. The sampler's mean negative logprob on its own sampled tokens (rollout log-ppl, bottom, computed for free during generation) reaches its minimum at step 76, rises monotonically after, and crosses 1.5x its running minimum at step 114, 38 steps before the length break. The dense control (orange) never inverts across 289 steps.

2. What moved in the weights: a uniform, informationless walk

Between steps 100 and 200 every 2D tensor moved at 0.38 to 0.39 lr per coordinate per step, from 128-element norm gains to 10M-element projections (399 tensors, std 0.025). Only the tied embedding is slower (0.28), because rows of absent tokens cannot move. Total relative movement was tiny: 0.15 percent.

Within tensors, realized per-coordinate movement is uncorrelated with the optimizer state (correlations with m, v, and the implied step size are all near zero on projection tensors). Movement direction in the collapse window is nearly orthogonal to the learning direction (median cosine 0.13). The walk carries no information about the objective.

3. The optimizer state is a flight recorder

Left: the median implied Adam step per coordinate clusters at ~0.6 lr at both checkpoints, a signSGD signature. Right: the momentum direction at step 100 and step 200 is fully decorrelated (median cosine 0.0005). The only persistent-direction outliers are early-layer q/k norm gains, which surgery later shows are functionally irrelevant.

A sharp invariant: the sum over all coordinates of v equals (1 - 0.999^t) to a relative error of 3e-8 at t=100 and 5e-7 at t=200. That is only possible if the post-clip global gradient norm was exactly at the 1.0 clip bound at every single step. The gradient magnitudes AdamW saw were pure ceiling, and v's apparent doubling is nothing but beta2 warmup. Global ||m|| is 0.64 against 0.23 expected for sign-incoherent streams, direct evidence in the state itself that update signs were held coherent across ticks.

4. The mechanism: a sign railgun

The correction applied to every floating tensor on every tick from 20 onward is G_corr = 0.25 G + 0.75 |G| sign(M). Where the masked gradient agrees with M nothing changes, and where it disagrees the result is 0.5 |G| sign(M). Either way the sign is sign(M). M refreshes only at anchor fires (every 20 ticks), so the sign pattern of the entire update stream is frozen between fires. An independent adversarial verifier reproduced this bitwise and confirmed from the run's own counters that the correction fired on exactly 398 tensors on every step from 20 to 200 with no per-tensor exemptions.

A per-coordinate simulation of AdamW under this stream (lognormal magnitudes, global clip, signs frozen in 20-tick windows, modest cross-window correlation) jointly reproduces the three measured statistics: implied step 0.72 vs 0.61 observed, realized speed 0.43 vs 0.39 lr per step, window-coherence ratio 0.46 vs 0.45 to 0.61. The same simulation with per-tick independent signs moves 4x slower. Freezing signs for 20 ticks is worth roughly 16x in diffusion rate.

5. Causal surgery: direction is everything

Teacher-forced entropy on 250 fixed MATH probes (higher = flatter = damaged). Adding matched-norm random noise to the healthy checkpoint does nothing. Adding the real delta's orthogonal component alone reproduces the full damage. Restoring norm gains or the embedding to the collapsed model does nothing. Restoring attention recovers the most.

model	TF entropy	greedy acc	greedy trunc	sampled acc (T=1)	sampled trunc
base	0.590	0.645	0.050	0.316	0.147
step 100 (healthy)	0.613	0.775	0.035	0.676	0.092
100 + matched random noise	0.626	0.770	0.070	0.676	0.098
step 200 (collapsed)	1.415	0.470	0.135	0.150	0.125
200 with 100's attention	1.053	0.620	0.035	0.336	0.055
200 with 100's MLP	1.200	0.525	0.225	0.258	0.305
200 with 100's layers 27-35	1.025	0.605	0.015	0.281	0.031
200 with 100's layers 0-8	1.707	0.330	0.460	0.102	0.410
200 with 100's embedding	1.426	0.455	0.115	0.152	0.125
200 with 100's norm gains	1.4154	0.455	0.165	0.117	0.164
100 + orthogonal part of delta (99% of norm, learning excluded)	1.420	0.410	0.205	0.148	0.102
100 + coherent part of delta (the learning direction)	0.626	0.755	0.100	0.676	0.113
100 + one synthetic 20-tick railgun kick (from the step-200 state)	0.743	0.740	0.020	0.645	0.039

Three conclusions. First, distance is not damage: a random direction of the same per tensor size is harmless (paired isotropic accuracy delta 0.000 against real -0.53, effect separation 58x with CI 49 to 72). Second, the damage is distributed and co-adapted: no small tensor group carries it, restoring healthy early layers into the collapsed model is actively harmful (the damaged downstream co-adapted to drifted upstream features), while the reverse insertion of damaged early layers into the healthy model costs almost nothing. Third, the damaging object is precisely the orthogonal churn the railgun leaves behind, and attention weights carry its functional bite disproportionately (twice their norm share).

Walking along the real delta, damage grows smoothly. There is no weight-space cliff. The temporal cliff at step ~150 is a property of the closed loop with the data distribution, not of the weights, which widens the early-warning window.

6. How flat weights become runaway lengths

On the healthy policy's own solutions, the healthy policy puts EOS top-1 at its stop position on 100 percent of trajectories (logprob -0.43). The collapsed model, on identical text, drops that to 11.6 percent (logprob -3.18) while entropy on the same tokens rises 11x.

Greedy decoding still terminates (the collapsed model's greedy length is 663), so validation greedily measured mostly a math-quality drop. But training samples at temperature 1.0: with per-position stop probability around 0.04, trajectories leak past their natural endpoints and wander, so mean length and the truncated fraction climb (0.05 at step 100, 0.55 at step 170), truncated rollouts poison the GRPO advantages, and the anchor's M starts chasing a degenerating batch. The 0.55-truncation state near step 170 was never checkpointed, so its internals are an inference from the mechanism, while step 200 itself is a partly shortened degenerate mode. This also explains the selectivity: multiple-choice likelihood comparisons (MMLU-STEM rose 22 points) survive flattening, sequential sharp decisions (termination, exact formatting) do not.

7. Red flags, with thresholds and lead times

flag	signal	threshold	fires at	lead vs onset 152	false positives	cost
RF1 primary	rollout log-ppl (sampler's own logprobs)	1.5x running min, 3 consecutive steps	~114	~38 steps	none in dense 289 steps, none in optreset 500	free during generation
RF2 watch	same	1.3x running min	~107	~45	one near-miss in optreset	free
RF3 confirm	response_length/clip_ratio	> 0.12 absolute, 3 steps	~150	0	none (dense max 0.05, optreset max 0.016)	free
RF4 structural	global \|\|exp_avg\|\| after step	> 0.45 (at clip bound 1)	~tick 40	regime flag, not onset	iid streams sit at ~0.23	one norm reduction
RF5 structural	per-coordinate speed from 2 snapshots 10-20 steps apart	uniform ~0.4 lr/step across classes	any window	regime flag	healthy streams are heterogeneous	CPU only

RF1 and RF3 need no validation run and no extra forward passes. RF4 and RF5 detect the pathological update stream itself, long before behavior degrades: in this run the regime was present from tick 40 while the score was still climbing to its 0.80 peak at 114.

8. Corrections, ranked

#	fix	defect targeted	evidence	falsifiable GPU test
1	Anchor-sourced optimizer moment reset (optreset)	v built from garbage magnitudes	PROVEN: the optreset50 arm is the only compressed run to survive 500 steps (val 0.7255 at 500, rollout ppl dense-like 0.042) with the same mask and railgun	cadence 20 vs 50, v-only vs m+v, scale-match ablation, 300 steps with RF telemetry
2	Value-blend correction: G = a G_comp (\|\|M\|\|/\|\|G_comp\|\|) + (1-a) M, staleness-decayed	the sign transplant itself + magnitude garbage	railgun algebra verified bitwise, sim shows iid stream diffuses 4x slower, dense-magnitude streams are proven safe (dense control)	300-step rerun: falsified if RF4 stays > 0.45 or RF1 fires
3	Rail decoherence: apply the transplant to a fresh random coordinate subset each tick	20-tick sign freeze	simulation: window coherence is worth 4x diffusion speed	same rerun, watch RF5 heterogeneity
4	Per-tensor magnitude normalization of G_comp to \|\|M\|\| scale	25x inflated magnitudes, clip pinned at bound	clip was binding at 1.0 on all 200 steps, erasing grad_norm as a signal	composable with 2 or 3, check clip stops binding
5	RF1 tripwire: on fire, drop the transplant until next anchor fire	guard, not cure	RF1 calibration above	inject at step 114 in a rerun, measure survival
6	Slow-channel sharpness restoration: anchor distills its own top-k logits into the fast policy on the existing dense cadence	flattening symptom	acts closest to the measured damage, most machinery	KL-to-anchor probe on burst checkpoints first

All six keep the compression, add no new communication class, act at the gradient/optimizer level (task-agnostic, nothing MATH-specific), and the top two keep dense-magnitude information flowing so learning speed is preserved. Symptom-level patches (length penalties, cap tuning, stop-token shaping) are excluded by construction.

9. Verification

Five independent adversarial verifiers, each instructed to refute one core claim by recomputation from the raw artifacts, returned five CONFIRMED verdicts with corrections that were adopted (exact clip semantics, embed-row frequency structure, scope of the runaway-leak inference, paired-statistics strengthening of the noise control, and the alpha < 0.5 transplant condition). The offline gauges reproduce the run's own validation numbers at all three anchors: base 0.641 vs 0.645, step 100 0.746 vs 0.750, step 200 0.443 vs 0.437.

10. Reproducibility

Scripts in research/scripts/weight_archaeology/: reconstruct_all.py (fp32 model + AdamW state from FSDP shards, 3 validated invariants), battery_ab.py, battery_c.py, battery_c2.py, battery_e.py (weights and optimizer forensics), probe_build.py, teacher_force.py, gen_gauge.py, tf_onpolicy.py (behavior gauges), build_variant.py + run_surgery_round{1,2}.sh (surgery and interpolation), make_figures.py, make_report.py. Box artifacts under /workspace on the archaeology box. GPU validation plan: HANDOFF_GPU_VALIDATION.md.