Max-Collab
Weekly working notes from the collaboration with Max Ryabinin on extending PipelineRL. Each entry is a self-contained readout for team members who were not in the meeting.
2026-07-22Discussion #1Streaming PipelineRL: Max's layer-wise weight updates
PipelineRL already trains while it keeps generating. Max proposes streaming each new weight version out of the trainer piece by piece, at pipeline-stage boundaries as backprop finishes them, instead of shipping and loading the whole model at once. Off-policyness does not get worse, and the win grows with model size up to pipeline-parallel scale. The first step is to measure the stall.