MOAT Projector Verdict — the ANCHOR/FAST weight projector, from GSM8K to Big-Math

Integration of five MOAT lanes · #45 (scorecard) · #47 (linear) · #48 (2nd-order) · #49 (adaptive) · #60 (cross-dataset) · Qwen2.5-1.5B-Instruct · fast CORE-4 harness, full-fidelity confirmed

1 · The question

In communication-efficient pipeline-parallel GRPO, a stale ANCHOR worker must cheaply stay close to the live model. The test: can it predict future weights more accurately than simply holding its stale copy? We score seven projectors — hold_stale (do-nothing), naive_linear, damped_linear, naive_second_order, damped_second_order, and the adaptive adaptive_linear/adaptive_second_order — plus the #49 self-correcting arms, replayed over real fp32 GRPO weight trajectories. The headline metric is weight_proj_ratio = projected-error / stale-error (a value < 1.0 means projection helps); absolute accuracy is pred_evr (explained variance vs the true future weights). Two datasets: GSM8K (EXP-57) and Big-Math (EXP-58).

2 · GSM8K — projection wins

At the operating point (per-step Δ=10, h=10), damped_linear is the winner: ratio 0.9396 with λ*=0.3. The adaptive arms edge it by hundredths (best arm armA K5 = 0.9351, only −0.004), and fixed second-order is far worse (1.1482, curvature zeroed out). Prefer-simplicity keeps the fixed rule.

Ranked by weight_proj_ratio at op-point (per-step Δ=10, h=10, GLOBAL row). Green row = the chosen fixed winner; blue = best adaptive arm. Source: MOAT-48 (base ladder) + MOAT-49 (arms).

Projection helps everywhere on GSM8K — every block type, super-block and special family sits below 1.0 (best: embeddings & tied lm_head at 0.918). No load-bearing family breaks, so a single global scalar is enough. R²↔ratio Spearman ρ=−0.75 (from #47): the more linearly-predictable a group's trajectory, the more projection helps it.

3 · Cross-dataset — Big-Math breaks it

Re-scored on Big-Math at its op-point (per-tick Δ=5, h=10 = 200 global steps), every projector collapses to do-nothing. The λ-grid drives the fixed rule to λ*=0.0 (ratio exactly 1.0000); the best adaptive arm gains a rounding-level −0.0001; naive projection blows up (1.7–4.2×).

Ranked by weight_proj_ratio at op-point (per-tick Δ=5, h=10, GLOBAL row). Do-nothing is effectively best; the fixed bar degenerates to it. Source: MOAT-58.

On Big-Math no family is helped (all λ*→0, ratio 1.0000) and bias is actively harmed (1.0078) — the one place projection does anything, it hurts. The mechanism is the alignment of consecutive weight updates: GSM8K's optimization path is coherent (0.86, a near-straight line to extrapolate), Big-Math's is not (0.15, each step turns a fresh direction) — so there is nothing for a linear rule to project along.

4 · The eight verdict questions

5 · Tensor-family verdict & decision rules

Required family comparison

Decision rules applied

6 · State-cost vs gain

ANCHOR should carry the smallest state that clears the bar. Adding scalars (adaptive coefficient, curvature) never buys ≥0.01 on GSM8K, so the 1-scalar fixed rule is the efficient choice.

7 · Failure mode & what it means for comm-eff ANCHOR

Why projection stops helping. Linear (and second-order) projection assumes the weight trajectory has a persistent direction — that the next Δ steps continue roughly where the last Δ pointed. That holds on GSM8K (consecutive-update cosine 0.86) and fails on Big-Math (0.15): when successive optimizer steps are near-orthogonal, the best linear extrapolation of a stale anchor is itself (λ→0), and any non-trivial projection only adds error. This is not a harness artifact — it reproduces at full tensor fidelity and is uniform across depth and block type.