verl comm-eff research · offline weight-projection · rank-1 trajectory · task-dependence test · 30-tick Big-Math
Does the rank-1 “one straight line” story survive a harder task? Big-Math vs GSM8K, same model & recipe
On GSM8K the RELEX picture held cleanly (see the GSM8K report):
an LLM’s RLVR weight update is almost perfectly rank-1 (moves along one fixed
direction), the distance along it grows linearly, and that direction is
locally stable. The operator’s hunch: maybe that is a property of an easy
task, and a harder task — where the model has more to learn and a messier optimization
path — would look less like one clean line. We re-ran the identical analysis on
Big-Math (EXP-58), same Qwen2.5-1.5B-Instruct, same GRPO, and put the
two side by side.
Verdict — the hunch is right, with a sharpened conclusion.
Rank-1 itself is not what breaks: on Big-Math one direction still captures the bulk
of every tensor’s move (EVR₁ 95.6% at an 8-checkpoint window). What
degrades on the harder task is the durability of that single line — how much
of the trajectory stays on it and how long its direction lasts. Over the same number of
checkpoints, 25.1% of Big-Math’s accumulated move lies off the
line vs only 4.5% on GSM8K (≈6× more), the
along-line clock is less linear (R² 0.939 vs 0.985), and the
direction rotates further from where it started
(|cos| 0.46 vs 0.64 over
each run). So: rank-1 is universal; “one fixed line for the whole run” is
task-dependent, and the harder the task the more it wants a recent, not global, line.
Read the magnitudes with one caveat. The two traces are sampled at
very different rates: GSM8K checkpoints are ≈0.5 optimizer steps apart, Big-Math’s are
20 steps apart (≈40× coarser). So a “window of W checkpoints” spans far more training
on Big-Math, which by itself lowers EVR₁ and raises the off-line share. We therefore compare
at a fixed number of checkpoints (the unit a staleness-compensator actually sees) and
normalize the drift curve to each run’s own length — but the size of every gap below
mixes “harder task” with “coarser snapshots.” The direction of every effect is
consistent and robust; treat the exact multipliers as indicative, not exact.
“Big-Math is…” colours green when the harder task looks better on that metric,
red when worse. Every stability/durability metric is worse; raw rank-1-ness is barely touched.
1 · Are the updates still rank-1? (yes — on both)
GSM8KBig-Math
Dashed lines = medians. Both tasks pile up near 1.0: over a fixed 16-checkpoint window one
direction still explains most of each tensor’s change (GSM8K 99.7%, Big-Math
89.9%). Big-Math’s mass is shifted left — a bit less rank-1 — but it is emphatically
not the thing a harder task destroys. The single-direction assumption survives.
2 · Does the distance grow linearly? (less cleanly on Big-Math)
GSM8KBig-Math
GSM8K median R² 0.985 (65.6% of tensors clear the paper’s
0.98 bar); Big-Math 0.939 (0.0% clear it). The “how-far-along”
coefficient still trends linearly on Big-Math but with visibly more wobble.
GSM8KBig-Math
Per module: the gap is broad, not driven by one outlier block — attention and MLP
projections alike are a little less linear on the harder task.
3 · Is it still one durable direction, or does it drift? (the real difference)
GSM8KBig-Math
v₁ estimated from a sliding window vs v₁ from the earliest window (|cos|=1 → identical
direction). Both start at 1.0; the steeper the fall, the more the “shared” direction rotates across
training. Over its own run Big-Math falls to 0.46 vs GSM8K’s
0.64 — the harder task’s single direction is markedly less durable.
GSM8KBig-Math
The cleanest same-#checkpoints view: what fraction of the accumulated move sits OFF the
single fitted line. At W=8, Big-Math 25.1% vs GSM8K 4.5% — the line
leaves far more of the harder task’s movement unexplained, and the gap widens with window size.
Why this is the crux. A rank-1 predictor assumes the
future move continues along one stored direction. If a large slice of the real move is already
off that line — and rising — then the stored line is a weaker guide to where the weights go next. On the
harder task that off-line slice is several times larger, which is exactly why “freeze one global
direction” degrades and a recent-window line is needed to keep up with the faster rotation.
weight_proj_ratio: below 1 beats holding the stale weights, above 1 is worse than doing
nothing. Anchored rank-1 hugs 1.0 on both (it describes position, not velocity). The sharp
contrast is momentum (naive_last2): a strong short-horizon winner on GSM8K but harmful on Big-Math
— largely because 20-step-apart checkpoints decorrelate consecutive deltas, so “last step ⇒ next step”
no longer holds. This panel is the most cadence-sensitive; read it as “coarse checkpoints kill naive
momentum,” which is itself a reason the harder/coarser regime needs the anchored line as the safe fallback.
5 · What this means for the comm-eff design
Rank-1 is a property of the optimizer/model, not the task — it replicates on Big-Math, so the
trajectory-clock state (v₁ + a couple of scalars per tensor) is still well-defined and cheap.
The task-dependent part is durability: the harder task’s trajectory leaves more of itself off the
single line and rotates that line further, so a frozen global line fits worse. The design
implication is the same as GSM8K’s and reinforced: use the rank-1 line as a cheap
position / reconstruction / regularization state maintained from a recent window, never as a
frozen global velocity predictor — and the harder the task, the shorter “recent” must be. Momentum-style
short-horizon repair that worked on GSM8K’s fine cadence does not survive coarse checkpoints, so on
harder/longer runs the anchored rank-1 line is the never-harmful fallback to lean on.
Provenance & honest limits
Data. Big-Math = EXP-58 fp32, 30 checkpoints (steps 20–800, 20-step cadence — the
first ~80% of the 1000-step run; the last 20% was not downloaded to respect local disk). GSM8K reference =
EXP-57 fp32, 160 ticks. Panel = layers {0,7,13,20,27} + final norm (61 matrices).
Cadence confound. The 40× snapshot-spacing difference inflates every fixed-window gap; we cannot
fully separate “harder task” from “coarser snapshots” without a matched-cadence GSM8K trace (its fp32 trace
is no longer on disk). Directions of effect are robust; magnitudes are indicative.
Method. Both scored by scripts/rank1_scorecard.py (identical metric contract, real-data
audit PASS at ~1e-6 on Big-Math); direction probe scripts/rank1_direction.py (unit-norm recon err
~1e-15); this page scripts/rank1_report_math.py. Structural metrics use fixed windows W∈{8,16} so
the delta-count matches across tasks.