Add RLVR-paper models + math datasets to comm-eff GRPO (additive, reward/prompt-validated, smoke-tested)

Close-out verdict (issue record)

VERDICT: PASS · 2026-07-07 · additive enablement, no algorithm path forked · PR https://github.com/shamanez/verl/pull/21 (merged)

All 4 stage gates GREEN (none skipped):

Result: 26/26 cells PASS. 5 test models (qwen25-math-1p5b, qwen3-1p7b-base, r1-distill-qwen-1p5b, open-nemotron-1p5b, qwen3-4b-base) × 5 math datasets (math, numina-cot, deepscaler, skywork-or1, dapo-math) all yield non-null NaN-free rewards through the reused comm-eff launcher (override MODEL_PATH + DATA_DIR only). All 5 route through DigitalLearningGmbH/MATH-lighteval → math_reward; NO new reward function. 3 integrate-only 7B/8B models: CPU tokenizer/chat-template load only (Stage 1).

Recorded negatives (clean = gate PASS): open-nemotron-1p5b near-zero reward on all datasets (weak at math-\boxed{}; non-null so kept as low-capability, not dropped). No OOM (incl. Qwen3-4B on 1×H200); no model dropped. R1-Distill ran at resp=4096, no truncation nulls.

Cost: operator-provided 1×H200 NVL (EXTERNAL). One disk-full retry (per-step FSDP checkpoints filled 200G; driver now runs SAVE_FREQ=-1 + per-cell checkpoint cleanup). Box torn down post-merge.

Detail: research/runs/62-rlvr-models-datasets/verdict.md, matrix_status.tsv, preflight.json. WandB group: verl_compression_research / 62-rlvr-models-datasets.

Analyst verdict (verdict.md)

Verdict — Issue #62: Add RLVR-paper models + math datasets to comm-eff GRPO

VERDICT: PASS (2026-07-07)

Additive enablement. All 4 stage gates evaluated, none skipped, all GREEN. No

algorithm path forked; every dataset routes through the locked

DigitalLearningGmbH/MATH-lighteval → math_reward key; no new reward function

added. Dense control byte-identical.

Stage gates

Stage 3 — 25 smoke cells (train/val reward @ 5 steps, all PASS, all train_rc=0 except one benign wandb-teardown rc=1)

Stage 4 — dense-control-parity

Qwen2.5-1.5B-Instruct, GSM8K, comm-eff OFF: train_rc=0, train reward 0.438, val 0.697 — sane/in-band.

Off-path byte-identical: verl/utils/reward_score/__init__.py, vast_comm_eff_accel_base_*.sh,

vast_comm_eff_baseline_*.sh, run_qwen3_4b_fsdp.sh all 0 changed lines vs base branch.

Recorded negatives (clean negatives = gate PASS, per plan)

• open-nemotron-1p5b scores near-zero on all 5 datasets (0.00–0.04) — weak at math-\boxed{}

output. NOT a failure: emits valid non-null, NaN-free rewards; the reward loop works. Kept in

the registry as a documented low-capability entry, not dropped.

• R1-Distill cells ran at MAX_RESPONSE_LENGTH=4096 (plan q5) — no truncation-driven null

rewards observed; all 5 passed.

• No OOM on any model (incl. Qwen3-4B-Base on 1×H200). No model dropped from the registry.

7B/8B integrate-only (not GPU-smoked, per plan)

r1-distill-qwen-7b, r1-distill-llama-8b, qwen3-8b-base: registry + CPU tokenizer/chat-template

load only (Stage 1, 8/8). Verified load a chat template preserving the boxed instruction.

Infra notes (not gate-affecting)

• First full-matrix run filled the box's 200G disk with per-step FSDP checkpoints (~20G/cell,

verl force-saves final step). Fixed: driver now runs SAVE_FREQ=-1 + rm -rf checkpoints

per cell; resume mode re-ran only the 18 failed cells. Peak disk after fix: 169G free.

• Box: operator-provided 1×H200 NVL (EXTERNAL). Model downloads amortized (each of 5 models

pulled once, reused across its 5 datasets).