latent
Faster isn't always better. Sometimes the right answer needs patience. That's the idea behind reasoning models — a class of LLMs that deliberately spend more compute at inference time to produce more accurate, more reliable answers. The trade is straightforward. You pay with latency and tokens, you get back correctness.
To see why this works you have to peel back a couple of layers of how language models actually generate text.
How standard LLMs work
A conventional large language model generates text through autoregressive prediction. Given a sequence of tokens, it predicts the next one. Then it appends that token and predicts the next one after that. One forward pass through the network per token. No backtracking, no revision, no second thoughts.
That's surprisingly effective for a lot of tasks. The model has billions of parameters encoding patterns from its training data, and a single forward pass through those parameters can produce fluent, coherent, often-correct text.
But there's a real limitation. Each token prediction gets exactly one forward pass of computation. Whether the model is answering "What color is the sky?" or "Prove there are infinitely many primes," it gets the same compute budget per token. The difficulty of the question doesn't change how much thinking the model gets to do.
For simple tasks that's fine. For complex multi-step reasoning it's a hard ceiling. The model has to encode the entire solution path into a single stream of token predictions, with no ability to explore alternatives, check intermediate results, or back out when a line of reasoning fails.
The test-time compute insight
Reasoning models flip that constraint. Instead of allocating fixed compute per token, they spend variable compute based on the difficulty of the problem. The mechanism is dead simple: before producing the final answer, the model generates an extended chain of reasoning tokens. Those "thinking" tokens are the model working through the problem, and more thinking tokens means more computation applied.
This is what researchers call test-time compute scaling. Just as scaling training compute (more data, more parameters, more training steps) improves model capability, scaling inference compute improves the quality of individual responses.
The interesting finding: test-time compute scaling has its own scaling laws. Doubling the thinking budget doesn't double the quality, but it produces consistent, measurable gains on hard problems. And the gains are biggest exactly where they matter — on problems that need genuine multi-step reasoning.
Watch the same model, with and without thinking
You don't have to take my word for it. Below are two real recorded runs of the same model (Gemini Flash) answering the same puzzle — make 24 from the numbers 2, 3, 10 and 10, each used exactly once. The only difference between the runs is one API parameter: the thinking budget. Nothing is simulated and no LLM is called when you press play; you're watching recordings of the actual runs, at their real pace.
Same model, same puzzle, one parameter changed. With thinking off, the model answers in about four seconds — fluently, confidently, and wrong: check its expression and count how many tens it uses. With thinking on, expand the thinking block and watch it try combinations, hit dead ends, and back out before landing on a valid answer. It takes ten times longer. It's also correct.
The failure mode in the first run is worth dwelling on. The model doesn't say something crazy — it produces an expression that really does equal 24, violating a constraint it just restated. That's single-pass generation in a nutshell: no room to check intermediate results against the rules. The second run spends its extra tokens doing exactly that checking.
Training reasoning into the model
Reasoning models don't fall out of standard pretraining. You can't just prompt a base model to "think harder" and expect systematic improvement. The reasoning behavior has to be trained in, usually through reinforcement learning.
The training loop looks roughly like this.
First, generate candidate reasoning chains. Given a problem with a known answer, the model produces multiple different reasoning attempts. Some land on the correct answer. Some don't.
Then evaluate the chains. This is where the reward signal comes in, and there are two main approaches. Outcome Reward Models (ORM) only care about the final answer — did the chain reach the correct conclusion? Simple to implement, sparse feedback. Process Reward Models (PRM) evaluate each step in the chain independently — did this step follow logically from the previous one? Denser feedback, more reliable reasoning, but you need labeled step-by-step data to train them.
Then update the model. Using the reward signals, you make successful reasoning patterns more likely and unsuccessful ones less likely. Over many iterations the model learns to produce reasoning chains that reliably reach correct answers.
Standard LLM:
Input → [Forward Pass] → Output
Reasoning Model:
Input → [Think] → [Think] → [Think] → ... → [Answer]
↑ ↑
Variable compute Final response
(reasoning tokens) (grounded in thinking)
The RL approach is what separates reasoning models from simple chain-of-thought prompting. When you prompt a standard model with "think step by step," you're asking it to use a reasoning format, but the model wasn't trained to reason well in that format. A reasoning model was explicitly optimized to produce thinking chains that lead to correct answers.
What thinking tokens look like
The internal reasoning of these models is genuinely interesting to read. You've already seen the real thing — expand the thinking block in the "Thinking on" replay above and you'll find the model enumerating candidate expressions and discarding them. Here's a cleaner, simplified example of the same behavior on a logic problem.
<thinking>
The problem asks me to determine if all squares are rectangles.
Let me recall the definitions:
- A rectangle has four sides and four right angles
- A square has four equal sides and four right angles
So a square satisfies all the conditions of a rectangle
(four sides, four right angles) plus an additional constraint
(all sides equal).
Wait, I should verify this is not a trick question. Are there
any geometric frameworks where this relationship does not hold?
In standard Euclidean geometry, a square is always a special
case of a rectangle. This is definitional.
I'm confident in the answer.
</thinking>
Yes, all squares are rectangles. A square satisfies every
requirement of a rectangle (four sides, four right angles)
with the additional constraint that all sides are equal in
length.
Notice the self-correction ("Wait, I should verify..."), the hypothesis testing ("Are there any geometric frameworks..."), and the confidence assessment ("I'm confident"). These patterns come out of the RL training process. The model learned that chains containing these behaviors are more likely to reach correct answers.
On harder problems the thinking chains get longer and messier. A math competition problem can produce thousands of thinking tokens, with the model trying multiple approaches, hitting dead ends, backing up, and trying again. It's expensive, and it's dramatically more effective than a single-pass answer.
Where reasoning models excel
Performance gains from reasoning models aren't uniform. They're biggest in areas that require genuine multi-step logic.
Mathematics is where reasoning models shine the most. Problems that need chained algebraic manipulations, casework, or proof construction see the largest gains. On competition math benchmarks like AIME, reasoning models score 2-3x higher than standard models of comparable size.
Code generation is the next big one. Writing correct code usually means reasoning about edge cases, data flow, and algorithmic correctness. Reasoning models produce code that's more likely to be correct on the first attempt, especially for algorithmic problems.
Scientific reasoning benefits a lot too. Graduate-level science questions that need multiple concepts applied in sequence get noticeably better with extended thinking.
Complex instruction following also improves. Tasks with multiple constraints ("write a function that does X, handles edge case Y, and runs in O(n) time") get better because the model can explicitly check each constraint during its thinking phase.
The trade-offs
Reasoning models aren't a free lunch. The costs are real.
Latency. A model that generates 2,000 thinking tokens before answering takes significantly longer to respond than one that answers immediately. For interactive applications where users expect sub-second responses, that can be a dealbreaker.
Token cost. Thinking tokens aren't free. You pay for them the same way you pay for output tokens. A response that costs $0.01 with a standard model might cost $0.10 with a reasoning model generating long thinking chains.
Verbosity in thinking. Not all thinking is productive. Reasoning models sometimes generate repetitive or circular chains, burning through tokens without making progress. People are working on it.
Overhead on simple tasks. If you ask a reasoning model for the capital of France, it'll still generate thinking tokens. They'll be short and the overhead small, but it's still unnecessary computation for a task that doesn't need it.
When NOT to use reasoning models
This is the part that gets overlooked when people get excited about reasoning. There are clear cases where reasoning models are the wrong tool.
Simple factual lookups. If the answer is a single fact the model either knows or doesn't, extended thinking adds latency without improving accuracy.
Creative writing. Reasoning tokens optimize for correctness. Creative tasks benefit from fluency, style, and unpredictability, not systematic verification.
High-throughput, low-latency applications. Chatbots, autocomplete, real-time classification. If you need to process thousands of requests per second with minimal latency, reasoning models are too expensive.
Tasks where the model is already near-perfect. If a standard model already hits 99% accuracy on your specific task, spending 10x more compute for reasoning won't meaningfully improve outcomes.
The right approach is to use reasoning models selectively, for tasks where they provide meaningful accuracy improvements, and use faster, cheaper models for everything else.
The future: adaptive thinking
The most interesting direction in reasoning models isn't making them think harder. It's making them think smarter. Adaptive reasoning, where the model dynamically adjusts its thinking budget based on input difficulty, is the natural next step.
Picture a model that takes a question, does a quick initial difficulty assessment, then decides how many thinking tokens to allocate. Easy questions get a few tokens of verification. Hard questions get thousands of tokens of deep reasoning. Average cost stays reasonable, hardest problems still get the compute they need.
Early work in this direction looks promising. Some approaches train a separate difficulty estimator that routes questions to different thinking budgets. Others let the model itself learn to terminate thinking early when it hits high confidence.
This adaptive approach is what makes reasoning models practical for a much wider range of applications. Instead of picking between "fast and sometimes wrong" and "slow and usually right," you get a system that automatically balances the trade-off based on each specific input.
The trajectory is pretty clear. Intelligence is becoming a variable you can dial up or down based on the requirements of each task. That's a different paradigm from the fixed-capability models we've been working with, and it changes how we should think about building AI systems.