OpenThoughts: Data Recipes for Reasoning Models
Authors:
Etash Guha,
Ryan Marten,
Sedrick Keh,
Negin Raoof,
Georgios Smyrnis,
Hritik Bansal,
Marianna Nezhurina,
Jean Mercat,
Trung Vu,
Zayne Sprague,
Ashima Suvarna,
Benjamin Feuer,
Liangyu Chen,
Zaid Khan,
Eric Frankel,
Sachin Grover,
Caroline Choi,
Niklas Muennighoff,
Shiye Su,
Wanjia Zhao,
John Yang,
Shreyas Pimpalgaonkar,
Kartik Sharma,
Charlie Cheng-Jie Ji,
Yichuan Deng
, et al. (25 additional authors not shown)
Abstract:
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for training rea…
▽ More
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for training reasoning models. After initial explorations, our OpenThoughts2-1M dataset led to OpenThinker2-32B, the first model trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as AIME and LiveCodeBench. We then improve our dataset further by systematically investigating each step of our data generation pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields our OpenThoughts3-7B model, which achieves state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Diamond - improvements of 15.3, 17.2, and 20.5 percentage points compared to the DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on https://openthoughts.ai.
△ Less
Submitted 4 June, 2025; v1 submitted 4 June, 2025;
originally announced June 2025.
Transformers Struggle to Learn to Search
Authors:
Abulhair Saparov,
Srushti Pawar,
Shreyas Pimpalgaonkar,
Nitish Joshi,
Richard Yuanzhe Pang,
Vishakh Padmakumar,
Seyed Mehran Kazemi,
Najoung Kim,
He He
Abstract:
Search is an ability foundational in many important tasks, and recent studies have shown that large language models (LLMs) struggle to perform search robustly. It is unknown whether this inability is due to a lack of data, insufficient model parameters, or fundamental limitations of the transformer architecture. In this work, we use the foundational graph connectivity problem as a testbed to gener…
▽ More
Search is an ability foundational in many important tasks, and recent studies have shown that large language models (LLMs) struggle to perform search robustly. It is unknown whether this inability is due to a lack of data, insufficient model parameters, or fundamental limitations of the transformer architecture. In this work, we use the foundational graph connectivity problem as a testbed to generate effectively limitless high-coverage data to train small transformers and test whether they can learn to perform search. We find that, when given the right training distribution, the transformer is able to learn to search.
We analyze the algorithm that the transformer has learned through a novel mechanistic interpretability technique that enables us to extract the computation graph from the trained model. We find that transformers perform search at every vertex in parallel: For each vertex in the input graph, transformers compute the set of vertices reachable from that vertex. Each layer then progressively expands these sets, allowing the model to search over a number of vertices exponential in $n_{\text{layers}}$.
However, we find that as the input graph size increases, the transformer has greater difficulty in learning the task. This difficulty is not resolved even as the number of parameters is increased, suggesting that increasing model scale will not lead to robust search abilities. We also find that performing search in-context (i.e., chain-of-thought) does not resolve this inability to learn to search on larger graphs.
△ Less
Submitted 16 March, 2025; v1 submitted 5 December, 2024;
originally announced December 2024.
