Give the model time to think

The answer equals 24. It also cheats. Asked to use 2, 3, 10 and 10 exactly once, the model finishes with (10 / 10) * 3 * (10 - 2). The arithmetic works, but count the tens: it uses three. A second recorded run, with thinking enabled, reaches (10 - 3) * 2 + 10, which satisfies both the arithmetic and the rules.

That makes a useful demonstration of extended thinking. Extra computation gave the model another opportunity to find a valid answer. The interesting part is what happened along the way: both runs tried to correct themselves, and the longer run repeatedly called invalid expressions correct before choosing a valid one. To understand what we're buying when we give a model more time, we need to look beyond the final answer.

Two runs, one small puzzle

The task allowed addition, subtraction, multiplication and division, with parentheses. Each of the four supplied numbers had to appear exactly once. It's a good miniature test because we can check the answer ourselves without trusting another model.

The demonstration identifies the model as Gemini Flash and compares thinking disabled with dynamic thinking. Its scene configuration specifies a budget of 0 for the disabled capture and instructs the operator to change that setting between recordings. The published event logs preserve the identical prompts and responses, but omit the exact model version, per-run request settings and token usage. They let us examine the results; they don't establish a fully reproducible controlled benchmark.

Two recorded responses to the same 24-game puzzle. Thinking off finishes with an expression that uses three tens. Thinking on finishes with (10 - 3) * 2 + 10, which uses each supplied number once. Expand its thinking summary to see the unsuccessful candidates and repeated checks as well as the valid result. Playback adjusts pauses for readability; its running time is not a latency measurement.

The thinking-off event log and thinking-on event log give us a more precise comparison:

Times are offsets from the start of each recorded event stream, not independently measured client request latency. The completion offsets differ by about 6.5 times. Neither recording includes the token counts or billing data needed to turn that difference into a cost comparison.

The shorter response is especially instructive. It catches 102 / 3 = 34, rejects an expression for reusing the number 2, and notices that (10 - 2) * 3 leaves a 10 unused. It plainly can check its work. It just fails to apply that checking consistently to its final answer.

The longer run finds the valid expression in a summary chunk at 14.339 seconds, then keeps going. It repeats checks and endorses other expressions that still use three tens. Its final answer is correct, but every additional second wasn't necessarily useful. The displayed summary even says it checked with a calculator, although the recording contains no tool-call event. That sentence is a model claim, not evidence of an external calculation.

What extra thinking actually buys

A language model generates tokens, pieces of text, using the text already available as context. In a simple autoregressive decoding loop, it runs the network to predict another token, appends it, and continues. A short direct answer leaves little room for intermediate work. A longer response can contain calculations, rejected ideas and corrections that later tokens can use.

A useful analogy is scratch paper. If someone asks you for a number immediately, you have to commit early. If you can write down a few possibilities first, you can notice that one uses a number twice. The paper doesn't make your arithmetic infallible. It gives you somewhere to carry out work before choosing your answer.

Reasoning models make that intermediate work an explicit part of producing a response. Generating more reasoning tokens uses more computation before the final answer. The extra budget usually expands the sequence of work; it doesn't require each individual token prediction to become a fundamentally different operation.

Ordinary models can also generate intermediate steps and revise earlier statements. Chain-of-thought prompting research demonstrated improvements from examples of step-by-step reasoning without retraining the model. Our thinking-off recording makes the same basic distinction visible: disabling a dedicated thinking mode does not prevent reasoning in the answer text.

Here is a simplified view of the difference in response structure:

Direct answer:
  Input → [Generate answer tokens] → Output

Extended reasoning:
  Input → [Reasoning tokens, including possible checks] → [Answer]
           ↑                                             ↑
      Additional computation                       Final response
      before the answer                            still needs validation

Researchers call spending computation during answering test-time compute. Longer reasoning is one approach. Generating multiple candidates and using a verifier to select among them is another. Snell and colleagues found that the best allocation depends on the problem's difficulty and the model's capabilities. Their math experiments support choosing how to spend the budget, rather than expecting a fixed increase in thinking to improve every response.

How training makes that work more useful

Giving a model more space is only part of the problem. It also needs to learn how to use that space productively. Reinforcement learning is one way to encourage useful reasoning: generate attempts, score them, then update the model so rewarded behavior becomes more likely. The DeepSeek-R1 research reports reasoning behaviors emerging through large-scale reinforcement learning, while its R1 training pipeline also uses supervised stages. There is more than one training recipe.

The scoring method matters. In our puzzle, a reward that checks only whether the expression equals 24 would accept the wrong answer. A useful outcome check must also count the numbers and enforce the allowed operations. The definition of success has to contain the whole task.

Two common approaches differ in where they provide feedback. An outcome reward evaluates the final result. A process reward evaluates intermediate steps, so an invalid move can receive feedback before the final answer. The latter needs suitable step-level supervision and a dependable evaluator. In OpenAI's process-supervision experiments, process supervision outperformed outcome supervision on the studied math tasks. That is evidence for the method under those conditions, not a guarantee that every step-scoring system will be better.

Over repeated training, these signals can make useful patterns more likely: checking a constraint, trying another approach, or recognizing that a candidate failed. They can also leave weaknesses if the reward misses something important. The puzzle makes that danger easy to see. A neat calculation and a valid solution are different acceptance criteria.

Reading the thinking display

It's tempting to read an expandable thinking block as a complete window into the model. Google's Gemini thinking documentation describes the exposed text as thought summaries, not the complete raw thoughts. The replay therefore shows what the service exposed, not a full internal trace.

The following is an illustrative explanation of the kind of intermediate work a model might produce. It is not another recorded run or evidence of a particular model's training:

<thinking>
The problem asks me to determine if all squares are rectangles.

Let me recall the definitions:
- A rectangle has four sides and four right angles
- A square has four equal sides and four right angles

So a square satisfies all the conditions of a rectangle
(four sides, four right angles) plus an additional constraint
(all sides equal).

Wait, I should verify this is not a trick question. Are there
any geometric frameworks where this relationship does not hold?
In standard Euclidean geometry, a square is always a special
case of a rectangle. This is definitional.

I'm confident in the answer.
</thinking>

Yes, all squares are rectangles. A square satisfies every
requirement of a rectangle (four sides, four right angles)
with the additional constraint that all sides are equal in
length.

The useful work here is connecting the definitions to the conclusion. Phrases such as "I should verify" and "I'm confident" don't establish that verification succeeded. In the actual puzzle replay, similar language surrounds both valid and invalid candidates. Read the steps for what they demonstrate, then check the result against the task.

Where the extra work can pay off

Mathematics provides clear examples because many answers are easy to score. In its September 2024 o1 report, OpenAI reported an average of 74% on AIME 2024 with one sample per problem, compared with 12% for GPT-4o. Using consensus across 64 samples raised the reported o1 result to 83%. Those are named model and evaluation settings, not a universal multiplier for equally sized models. They also illustrate two different ways of spending more computation: reasoning within an attempt and comparing multiple attempts.

For code, the corresponding opportunity is work that benefits from examining edge cases and checking an algorithm before returning it. Scientific problems can require several concepts to be connected correctly. Complex instructions can require checking several constraints together, much like remembering both the target value and the number count in this puzzle. These are sensible workloads to evaluate; a benchmark result doesn't establish the improvement for your application.

The cost has several parts. Longer generation can delay the answer, and thinking tokens can increase the bill. Google's documentation says Gemini charges for thinking tokens and exposes their usage separately from the thought summary. You can't estimate the full bill by counting the words in the visible thinking block.

For an illustrative calculation, suppose a response costs $0.01 without extended reasoning and $0.10 with it. That tenfold increase is worthwhile only if the improvement matters enough to pay for it. Those figures are hypothetical, not prices or measurements from these recordings. Failed answers, retries and human review also belong in the comparison.

Simple factual lookups are a useful place to start with a fast baseline. Extra reasoning won't supply a missing source. For creative writing, judge whether it improves the actual draft; a math score says little about rhythm or originality. High-throughput classification and autocomplete need their own latency and quality measurements. Even a task already at 99% accuracy deserves a decision based on the remaining errors: removing a few expensive failures may matter more than the percentage suggests.

Let the task set the budget

Adaptive thinking is already available in the Gemini 2.5 API: setting thinkingBudget to -1 enables dynamic allocation based on the request. Model support and controls differ, so check the documentation for the specific model. Dynamic allocation can make reasoning easier to deploy, but it still needs evaluation against your workload.

For this puzzle, the longer run produced a valid final expression and the shorter run did not. That's a concrete success worth understanding. Two selected runs cannot tell us how often the result would repeat, and the longer summary shows why additional checking language should never be confused with a correctness guarantee.

I would test extended thinking where the task has several dependent steps or easily missed constraints, then compare repeated results against the same acceptance rules. Record latency and token usage alongside success. When an answer can be checked cheaply with code, use that check too. Giving the model scratch paper can help it find the solution; checking the arithmetic and counting the tens tells you whether it did.