MIT researchers double the training speed of reasoning models
A research group around Song Han at MIT, together with NVIDIA and ETH Zurich, has published a method that significantly accelerates the training of reasoning models — without any loss in accuracy. In the paper Taming the Long-Tail (arXiv 2511.16665, ASPLOS 2026), the authors report speedups of 70 to 210 percent over the state of the art. The official MIT press release is dated 26 February 2026, and the system is publicly available on GitHub under the name fastrl.
What the study shows in concrete terms
- 1.7× to 3.1× speedup in reinforcement-learning training of reasoning models, depending on model size and task.
- No accuracy regression — trained models reach identical benchmark numbers compared with conventionally trained baselines.
- Free byproduct: the “Adaptive Drafter” co-trained on the side can be deployed directly for speculative decoding during inference.
- Code is open: released at github.com/mit-han-lab/fastrl.
What was true before
Reasoning models like DeepSeek-R1, o3, or Claude reasoning variants are typically post-trained with reinforcement learning. In each RL iteration, the model itself generates solution paths (“rollouts”), which are then scored and used for the update. The problem is well known in the field: rollout lengths are extremely uneven. A few very long answers (the “long tail”) dominate the runtime of every step. While the short rollouts have long since finished, hundreds of GPUs sit waiting on the last few — physically idle.
Previous answers to the problem were either hard truncation (accuracy loss) or additional hardware investment (cost explosion).
What is true now
1. The Adaptive Drafter trains in the gaps. Hu, Yang, Guo, and Han use exactly the idle time during which most GPUs wait on the long-tail rollouts. They route a lightweight drafter model into co-training on those GPUs. The drafter learns in parallel to predict the main model — its training is literally free, because the GPU time would otherwise be wasted.
2. Speculative decoding accelerates subsequent rollouts. Once the drafter is good enough, the system places it in front of the main model as a speculative decoder. The drafter proposes several tokens ahead, the main model verifies them in a single pass. Combined with an Adaptive Rollout Engine that keeps pre-compiled CUDAGraphs for different batch profiles, the long tail measurably shrinks.
3. No model modification required. The technique is orthogonal to the training algorithm: anyone running GRPO, PPO, or a custom RL variant can drop in TLT/fastrl. End-to-end speedup measured on Qwen models is 1.7× in the most conservative setup and rises to 3.1× in configurations with a particularly pronounced long tail.
Why it matters
The result is interesting for two reasons — beyond raw training efficiency.
First, it shifts the cost equation for reasoning models. Today, the bulk of training cost lies in the RL phase, not in pretraining. Saving 70 to 210 percent here lets researchers run substantially more iterations per hardware budget — and that is exactly the bottleneck that today separates small research groups from well-funded labs.
Second, the method shows how closely training and inference optimization are converging: the drafter created during training becomes a productive inference accelerator. That is effectively a double lever on identical hardware. Comparable approaches in industry currently run as two separate pipelines.
What you can do now
If you train reasoning models yourself: look at the fastrl repo and check whether your RL pipeline is compatible. The integration is designed as a layer on top of existing RL frameworks and should run without architectural changes.
If you only consume reasoning models: watch whether your providers (Anthropic, OpenAI, DeepSeek, Qwen) lower inference latency or token prices in the coming months. Methods like this typically flow into production stacks within 6–12 months and become visible there.
If you follow research: the paper is a good example of how hardware-aware training (CUDAGraphs, memory layout, idle-GPU usage) is again becoming a central research field — after years in which attention focused almost exclusively on model architecture and data quality.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarDPO (Direct Preference Optimization)
DPO is a fine-tuning method that aligns language models directly to human preference pairs — without a separate reward model and without reinforcement learning.
LexikonFine-Tuning Explained — When, How, With What
When fine-tuning pays off, which methods exist (SFT, DPO, LoRA, QLoRA), and what AMD vs. NVIDIA hardware actually means in practice.