New Data-Selection Method Cuts LLM Training Time by Up to 62%

For the past two years, the answer to "how do we make models better?" has been depressingly simple: spend more. A new paper on arXiv makes the case for a different kind of answer — one that turns the economics of reasoning-model training on its head. The method, which combines two techniques the authors call difficulty-targeted online data selection (DOTS) and rollout replay (RR), slashes the wall-clock time of reinforcement-learning fine-tuning by 23% to 62% across six model-and-dataset combinations, while landing at exactly the same final performance as the standard GRPO recipe that has become the default for training reasoning models10.

What the new method actually does

The paper targets the most expensive phase of modern LLM development: RL fine-tuning on reasoning tasks, the stage where models like today's math and coding champions learn to think through problems step by step. The authors point out that this stage is resource-intensive in two distinct ways — it requires many training steps, and each step is expensive because the model has to generate long chains of sampled solutions, or "rollouts," to learn from. Existing work has largely ignored data efficiency in this setting, so the researchers attack both costs separately10.

The first technique is a curriculum, but an unusual one. Instead of sorting problems by some fixed human notion of difficulty, DOTS estimates what the authors call "adaptive difficulty" — how likely the current version of the model is to get a question wrong right now. A question the model always answers correctly teaches it nothing; a question it never answers correctly is equally useless. The sweet spot is moderate difficulty, where the learning signal is most informative, so the method prioritizes training questions whose predicted difficulty sits closest to a probability of 0.510.

Computing that difficulty directly would require generating full rollouts for every training question, which would erase the savings. The fix is an attention-based predictor: at each step, rollouts are generated only for a small reference set of questions, and the difficulty of the remaining questions is inferred from their similarity to that reference set. The second technique, rollout replay, is borrowed from classical RL's experience-replay playbook — a bounded first-in-first-out buffer stores recent rollouts and reuses them to fill out each training batch, cutting the number of fresh, expensive generations per step10.

The benchmark results, in detail

The headline numbers are worth reading closely. Across six combinations of Qwen2.5 models (1.5B, 3B, and 7B math variants) and reasoning datasets (MATH, DeepScaleR, ORZ, DeepMath), DOTS+RR reached the same final performance as the original GRPO baseline with 13.33% to 56.67% fewer training steps. Rollout replay added a consistent 11%–13% reduction in per-step time on top. The combined effect averaged a 40.7% cut in total training time, peaking at 61.65% on Qwen2.5-3B trained on DeepMath10.

Just as telling is the shape of the learning curves: the method doesn't merely reach the same destination faster, it holds higher accuracy at almost every point along the way, and the advantage persists when training is extended to 100 steps on the two settings tested10. The authors also validated the underlying mechanism directly, reporting high Pearson correlation between the attention-based difficulty predictions and the ground-truth difficulty measured by actual rollouts10. This is not a one-off benchmark win — it is a claim that the method changes the trajectory of learning itself.

Why this lands at exactly the right moment

The timing is what makes this paper more than an academic curiosity. MLCommons introduced its first LLM post-training benchmark in MLPerf Training v6.1 in late September, formalizing an agentic reinforcement-learning workload in which systems race to teach a 397-billion-parameter model to repair real software projects to a fixed pass@4 quality target of 0.695. When the industry's premier benchmarking body decides that RL post-training is now a first-class, timed workload, it is an implicit admission that this phase has become a dominant cost center. A method that cuts that phase nearly in half is not a marginal optimization; it changes what a fixed GPU budget buys.

The broader context reinforces the point. DeepSeek's R1 became the emblem of efficient reasoning training in early 2025, with a peer-reviewed Nature paper confirming its reasoning was trained for roughly $294K on top of the ~$6M spent on the V3 base model2. That result started an industry-wide hunt for ways to compress the reasoning-training bill, and the arXiv efficiency literature has been flooding ever since.

A crowded field of efficiency tricks — and how they fit together

Reading across the recent machine-learning literature, the new data-selection method is one strand of a wider pattern: efficiency research is splitting into complementary approaches that attack different resources. The DOTS+RR paper is data-centric — it asks which examples and which rollouts deserve compute. Its closest cousin in spirit is a two-stage recipe for mathematical LLMs published in July, which found that prolonged supervised fine-tuning (up to 10 epochs) pushes accuracy to its ceiling, after which a GRPO phase's real job is compressing solution length — improving token efficiency while preserving peak accuracy, with gains that grow with model size on AIME 2024 and 202512. Both papers converge on the same insight from different directions: GRPO-style RL is primarily an efficiency instrument, and treating it as such — rather than as a brute-force accuracy booster — is where the savings live.

A second strand attacks memory. A May benchmarking survey of parameter- and memory-efficient pretraining found that with well-tuned optimizers, full-rank training still wins on perplexity, but that two new techniques — weight refactorization and momentum reset — let a low-rank method on a 1B model beat popular memory-efficient algorithms like GaLore and Fira while using about 25% less memory9. A more radical December paper proposes reversible LLM architectures derived from hyperbolic differential equations, which reconstruct intermediate activations during backpropagation instead of storing them, enabling roughly 10× larger batch sizes, up to 101% throughput gains on a 96-layer model, and a lightweight fine-tuning procedure that converts existing checkpoints to reversible ones in about two epochs13.

A third strand, curriculum-guided layer scaling (CGLS), grows model depth in tandem with data difficulty during pretraining, outperforming compute-matched baselines on PIQA and ARC at both 100M and 1.2B parameter scales16. What unites all of these with the new DOTS+RR work is a philosophical shift: the frontier of efficiency gains is no longer only in hardware or parallelism, but in choosing when a model learns what, and how much of its own output it needs to generate to learn it.

The reading that matters

There is a temptation to file this paper under "incremental trick." That would be a mistake. The deepest claim in it is that the model's own competence, measured continuously during training, is the best guide for allocating data — an "adaptive" curriculum requiring no external supervision or human difficulty labels, which is what makes it scale10. Where the divergences in this literature appear is in emphasis: the memory-efficient-pretraining survey cautions that full-rank training still delivers the best raw performance when compute is unconstrained9, which is a fair warning that these techniques buy efficiency, not free capability. The DOTS+RR results sidestep that objection for now, since they match the baseline exactly rather than trading accuracy for speed — but the tests top out at 7B-parameter models on math reasoning, and a 40% saving at 3B is not guaranteed at 400B.

The open question is whether labs building frontier reasoning models will adopt the approach. The incentives point yes: RL post-training is now a formally benchmarked, timed workload5, reasoning models burn thousands of thinking tokens per query at inference6, and the most successful recent reasoning model won on cost discipline2. A method that gets the same reasoning performance from roughly half the RL compute is the kind of result that quietly reshapes training budgets. The authors' stated hope — that it encourages "data-centric approaches to improving LLM RL fine-tuning"10 — reads less like a modest conclusion and more like a prediction about where the field is going next.