Post-training field note Research briefing | 30 July 2026

Train specialists separately. Distill them back into one model.

That is the central pattern in Kimi K3 and DeepSeek-V4. Domain and reasoning effort are separate specialization axes. GLM uses a related idea, but its teachers are snapshots from sequential training stages.

Kimi K3 post-training: 9 teachers -> 1 student

Each teacher is a separate full policy specialized by domain and reasoning effort. MOPD selects the matching teacher for each sampled domain-effort pair and transfers its token distribution into one trainable student.

SFT checkpoint Verified trajectories establish the common cold-start policy.

General tasks Knowledge, reasoning, vision, faithfulness, search

lowhighmax

General agents Long-horizon assistant, research, writing

lowhighmax

Coding agents SWE, coding, kernels, web development

lowhighmax

Select teacher (d,e) Sample domain d and effort e, then use the corresponding policy teacher π_teacher^(d,e).

Update student π_θ The dense token-wise loss signal trains one effort-conditioned policy. This student becomes the unified K3 target.

MOPD dense token-wise loss signal, Equation 15

r_OPD^d(y_t | e,x,y_<t) = clip( sg( log π_teacher^(d,e)(y_t | x,y_<t) π_θ(y_t | e,x,y_<t) ), -R_max, R_max )

What is r? r_OPD^d is the dense token-wise loss signal, one scalar for every output token y_t. Its sign and size show how much more or less probability the matched teacher assigns than the student.

What is R_max? A positive clipping cap. It limits |r| to R_max so no single token creates an extreme signal. Kimi does not disclose its numeric value. §4.1.3, pp. 13-14

What exactly is the student? π_θ is the single trainable, effort-conditioned unified policy. It is distinct from the 9 teacher policies. The report does not disclose its initialization, whether the post-RL supervised pass initializes it, or the exact source of every training prefix.

How GLM and DeepSeek differ from Kimi

GLM-5.2 lineage

Sequential stages, then cross-stage recovery

§3, pp. 10-14

Overall SFTGeneral, reasoning, coding + agent.

Reasoning RLGRPO + IcePop; 4-domain mixture.

Agentic RLAsync coding + search rollouts.

General RLCorrectness, style, task quality.

Cross-stage OPDEarlier checkpoint logits recover lost skills.

Final policyOne model with turn-level thinking control.

DeepSeek-V4

Broad specialist bank, full-logit consolidation

§5.1, pp. 28-34

Per-domain SFTSeparate high-quality domain branches.

Per-domain GRPODomain rewards + separate effort configurations.

>10 teachersPhysically separate policy checkpoints.

Full-vocabulary OPDWeighted reverse KL on student trajectories.

Unified studentOne final policy for each model line.

Why distill, and is it OPD?

Yes, all three name an on-policy distillation stage. This is capability consolidation, not small-model compression. The teacher is not copied into the student by averaging weights.

A common offline form Fixed data or teacher-generated traces are prepared first; the student imitates the teacher on that fixed training distribution.

OPD used here The student rollout or response prefix defines where learning happens. The relevant teacher supplies logits or a teacher/student token-wise distillation signal on those visited states.

Not model merging Weight merging would combine parameters from checkpoints. Kimi and GLM do not report it. DeepSeek explicitly says OPD avoids degradation seen with traditional weight merging and mixed RL.

OPD algorithms only
Kimi K3	GLM-5.2 lineage	DeepSeek-V4
Route by (d,e), then clip sampled-token sg(log(π_teacher / π_student)) to [-R_max, R_max].	Use sampled-token sg(log(π_teacher / π_student)) in place of GRPO advantage; group size 1.	Minimize weighted full-vocabulary KL(π_student \|\| π_teacher) on student trajectories.

1 / 2

Compare, then build Research team shareout

Where the recipes actually differ

Short version: Kimi is the clearest domain x effort grid, DeepSeek scales the teacher bank, and GLM uses sequential checkpoints to repair forgetting.

Decision	Kimi K3	GLM-5.2 lineage	DeepSeek-V4
Specialists	Parallel: 3 domains x 3 efforts = 9 teachers.	Sequential: Reasoning RL, Agentic RL, then General RL.	Parallel: per-domain SFT + RL, then >10 teachers.
SFT	Prior-Kimi agent trajectories, verification, human annotation, XTML.	General, reasoning, coding + agent; rejection sampling and masked bad spans.	High-quality domain SFT; data sizes and mixture are not disclosed.
RL and rewards	K2.5-derived algorithm; verifiers + tournament GRM; partial rollouts.	GRPO + IcePop for reasoning; async agent RL; rules + ORM + GRM for general.	GRPO specialists; tests for verifiable tasks; rubric-guided GRM otherwise.
Reasoning effort	Max first, then budget annealing to high and low.	Per-turn thinking control; no disclosed 3-level RL effort ladder.	Non-think/high/max via different RL penalties and context windows.
Why distill	Unify the 9 domain-effort teachers.	Recover capabilities degraded by sequential objectives.	Unify >10 experts and avoid mixed-RL or weight-merge degradation.
Model merging	No parameter merge reported.	No parameter merge reported. Figure 5 uses teacher logits and a separate General RL weights path.	Explicitly not traditional weight merging. Uses logits-level alignment.
Final checkpoint	One unified K3 target + separate EAGLE-3 draft.	Output of cross-stage OPD; selection rule not disclosed.	One unified student; teacher weighting and selection rule not disclosed.
DPO / classic RLHF	Neither is reported as a stage.	Neither is reported as a stage.	No DPO; conventional scalar RLHF rewards are replaced by GRM scoring.

Reasoning RL, Agentic RL, and General RL are not the same thing

GLM is the only report here that names all three as sequential stages: Reasoning RL -> Agentic RL -> General RL -> cross-stage OPD. Kimi and DeepSeek cover similar capability families through specialist domains instead.

Reasoning RL

Optimizes solving problems with a checkable outcome: math, science, code, and tool-integrated reasoning. Rewards are usually tests, rules, or binary outcome judges.

Mapping: explicit GLM stage; inside Kimi's general-task domain; math/reasoning specialists in DeepSeek.

Agentic RL

Optimizes multi-step action: planning, tool calls, search, SWE, terminal work, and recovery from environment feedback. Rollouts are long, stateful, and often asynchronous or resumable.

Mapping: explicit GLM stage; Kimi general-agent and coding-agent domains; DeepSeek agent and coding specialists.

General RL

Optimizes broad response quality: correctness, instruction following, factuality, emotional intelligence, writing, role-play, and translation. Rewards mix rules, ORMs, GRMs, and human style anchors.

Mapping: explicit GLM stage. Kimi's general-task expert is only a partial analogue; DeepSeek reports instruction-following specialists, not a named General RL stage.

What the papers actually say about trainer efficiency

None says that the entire post-training run is low precision. The verified claims are selective: particular weights, activations, rollout engines, or stages use lower precision, while scheduling and state reuse remove the largest stalls.

Kimi K3

QAT starts at SFT; MXFP4 internal MoE weights + MXFP8 activations. RL rollout uses the same scheme.
Partial rollouts pause stragglers and resume them later.
Million-token KV/KDA and sandbox states are saved and resumed instead of recomputed.

GLM lineage

INT4 QAT is stated for SFT only. Rollout inference later uses FP8.
Async Agentic RL separates generation and training GPUs; TITO and direct importance sampling control drift.
MTP and prefill-decode disaggregation reduce long-tail latency; OPD uses group 1 and batch 1,024.

DeepSeek-V4

Post-training QAT covers teachers/reference/student; native FP4 is used for rollout and inference.
Teacher weights are loaded on demand; last-layer states are cached instead of full logits.
Token write-ahead logs resume interrupted rollouts without length-biased regeneration.

What the research team can learn from these pipelines

Separate domain and effort as two control axes

A coding expert and a max-effort expert solve different problems. Kimi makes the full cross-product explicit.

Distillation is the integration stage

RL creates specialists. OPD is what turns them into one deployable policy and, for GLM, restores capabilities lost later in training.

Use logits, not parameter averaging

All three transfer behavior through teacher distributions. DeepSeek explicitly prefers this to traditional weight merging.

Match the reward to the task

Use tests and state verifiers where possible. Use rubric-guided GRMs for work that cannot be checked deterministically.

Trainer efficiency is mostly a systems problem

Selective low precision helps, but resumable rollouts, KV and sandbox reuse, async generation, and teacher scheduling are just as central.

Important choices are still undisclosed

Teacher mixture weights, student initialization for Kimi/DeepSeek, and final checkpoint-selection rules are not fully specified.

2 / 2