Weekly Updates
Current and historical weekly readouts for the communication-efficient RLVR work.
2026-08-16Anchor signal reuse builds the collapsing bias, and controlling it removes the collapse without an optimizer-state swap
A weight-level autopsy shows the collapse is a small directional drift, about 0.1 percent of weight movement, driven by a correction held frozen between anchor refreshes. A family of controlled designs tests the hypothesis. Every design that reuses the frozen signal at full strength collapses or degrades, and applying it once per refresh removes the collapse while matching the optimizer-state swap, at no extra cost. The residual form used once is the best result of the program at +7.0 points.
2026-08-09Checking the two-circuit system at scale, and working out what happened
The two-circuit system taken up the scale ladder: an agentic multi-turn task at 1.5B, then 4B, then 8B with longer context windows. Where the runs break, what the accumulating sampler-trainer error does to the weights, and a first test of an idea for correcting the weight-space distortion by swapping the trainer's optimizer state from the anchor.
2026-07-24Two-circuit pipeline-parallel RL at longer horizons
The two-circuit method taken into hours-long runs: what keeps it stable over that horizon, why a rising KL is not the same thing as a widening train-inference gap, and how the compressed run lands on par with uncompressed RL at step 600.
2026-07-20Communication-efficient RLVR: the phase-one conclusion
The compressed method learns to full strength in and out of domain and matches an uncompressed run at saturation. On these datasets the collapse risk is manageable, and phase one closes with one open problem: a train and inference mismatch that compression enlarges, reads out as a rising KL, and has to be answered before this technique meets realistic asynchronous RL.
2026-07-13The two-circuit method beyond GSM8K: harder math and correct base models
On harder math the projection lever is worthless but the two-circuit correction holds; on a correct math-distilled base with 16k reasoning, both variants hold the dense line. Training only one middle block matches full-parameter RL in-domain.
2026-07-06Weight projection for two-circuit-network RLVR stability at longer K
Coherence-gated projection: GSM8K benefits, Math collapses to do-nothing at operational staleness.