Paper 651: Does Your Reasoning Model Implicitly Know When to Stop Thinking?

1. Paper Background

Extended-thinking language models like DeepSeek-R1 and Qwen-QwQ produce long internal reasoning traces before giving a final answer. The paper asks a deceptively simple question: do these models know when they have already found the answer and should stop, or do they continue reasoning well past the point of correctness?

The RFCS Metric

The paper introduces RFCS (Ratio of First Correct Step), defined as:

RFCS = (index of step where correct answer first appears) / (total number of steps)

An RFCS of 1.0 means the model reached the correct answer only at the very last step. An RFCS of 0.4 means the correct answer appeared after 40% of steps, with 60% of reasoning being redundant. The paper reports that in over 50% of correct responses RFCS is strictly below 1.0, confirming that over-thinking is widespread.

The SAGE Algorithm

The paper proposes SAGE (Self-Aware Guided Efficient Reasoning), a decoding algorithm that uses cumulative log-probability to score partial reasoning traces. Higher-probability prefixes correspond to the model being more confident in its current trajectory. SAGE selects the shortest high-confidence path found across a beam of candidates, reducing token usage while improving accuracy on MATH-500.

SAGE is implemented on top of TSearch (token-wise beam search) and can also be integrated into reinforcement learning fine-tuning via SAGE-RL (using GRPO/GSPO). On DeepSeek-R1-Distill-7B with MATH-500, the paper reports 92% accuracy with 2,213 tokens on average, compared to 84% accuracy and 3,419 tokens for the random-sampling baseline.

2. Model Substitution Rationale

The paper uses DeepSeek-R1-Distill-7B and Qwen-QwQ-32B. Both require local GPU deployment and are not available via a public API at cost compatible with this reproduction. We substitute Claude Haiku 4.5 (claude-haiku-4-5-20251001) via the Anthropic Messages API with extended thinking enabled.

3. Claims Being Reproduced

4. Quantitative Results

5. Charts and Distributions

6. Detailed Methodology

Dataset

We load the MATH benchmark from the Hugging Face dataset hub (lighteval/MATH, split: test, config: all). This is the same 500-problem benchmark used in the paper. We sample 50 problems uniformly at random with seed 42. Each problem includes its difficulty level (1 through 5) and a LaTeX solution from which the ground-truth answer is extracted using a nested-brace boxed parser.

Answer Extraction

We parse the final answer from the last boxed expression in the response. The parser handles nested braces by tracking depth, so expressions like boxed{x = frac{1}{2}} are parsed without truncation. Normalisation strips dollar signs, commas, whitespace, and converts to lowercase before comparison.

Step Segmentation

Following the paper (Section 2, citing Chen et al. 2025), reasoning steps are delimited by double newlines. Each non-empty segment is treated as one step. We compute RFCS by replaying cumulative prefixes: after each step, we check whether the current prefix contains a correct boxed answer. The step index of the first correct prefix, divided by total step count, is the RFCS value.

Novel Analysis 1: Redundant Step Classification

For each correct response with RFCS below 0.95, we extract the steps occurring after the first correct step and send each (truncated to 800 characters) to Claude Haiku as a single-token classifier. The classifier is given a fixed taxonomy of five categories and must return exactly one label. We aggregate counts across all responses and all problems.

Novel Analysis 2: RFCS by Difficulty

Each MATH problem carries a difficulty field ("Level 1" through "Level 5"). We parse the integer level and accumulate RFCS values per level, then compute per-level means. We also compute Pearson correlation between difficulty level and mean RFCS to test the directional hypothesis that harder problems produce lower RFCS (more redundancy).

Novel Analysis 3: Correctness Trajectory and Flip Detection

For each response, we build a boolean trajectory by replaying step prefixes and recording whether the current running answer is correct. We count the number of transitions in this trajectory (flips). We separately count dangerous flips: transitions from True to False that occur after the first correct step, representing cases where the model discards a correct answer and replaces it with a wrong one.

Thinking Budget Ablation

We run the first 10 problems from our sample at three thinking budget sizes: 2,000, 6,000, and 12,000 tokens. For each budget we measure accuracy, average RFCS, and average thinking length in words. This tests whether more compute consistently improves accuracy and how it affects redundancy.

7. Example Problem Walkthroughs

The following panels show summary data for problems where the model's answer changed direction during reasoning, illustrating the flip phenomenon described in Novel Finding N3.

8. Thinking Budget Ablation

9. Agent Code

def compute_rfcs(thinking, correct_answer):
    steps = [s.strip() for s in thinking.split('\n\n') if s.strip()]
    if not steps:
        return None
    cumulative = ""
    for i, step in enumerate(steps):
        cumulative += step + "\n\n"
        if answers_match(extract_boxed(cumulative), correct_answer):
            return (i + 1) / len(steps)
    return None

def correctness_trajectory(thinking, correct_answer):
    steps = [s.strip() for s in thinking.split('\n\n') if s.strip()]
    trajectory, cumulative = [], ""
    for step in steps:
        cumulative += step + "\n\n"
        trajectory.append(answers_match(extract_boxed(cumulative), correct_answer))
    return trajectory

def count_flips(trajectory):
    return sum(1 for i in range(1, len(trajectory))
               if trajectory[i] != trajectory[i - 1])

CLASSIFIER_SYSTEM = """You classify reasoning steps from a math solver.
The model has ALREADY reached the correct answer in a prior step.
Classify the given step into exactly one category:

verification       - re-checking or confirming a result already established
re-derivation      - solving part of the problem again with the same method
alternative-method - attempting a different solution approach from scratch
meta-commentary    - thinking about the reasoning process itself
conclusion         - writing out the final answer statement
other              - anything else

Reply with ONLY the category name, nothing else."""

def classify_step(step_text):
    resp = client.messages.create(
        model=MODEL, max_tokens=20,
        system=CLASSIFIER_SYSTEM,
        messages=[{"role": "user", "content": step_text[:800]}],
    )
    label = resp.content[0].text.strip().lower()
    return label if label in STEP_CATEGORIES else "other"

ABLATION_BUDGETS = [2000, 6000, 12000]

def thinking_budget_ablation(problems):
    subset = problems[:10]
    results = []
    for budget in ABLATION_BUDGETS:
        rfcs_vals, accs, tok_lens = [], [], []
        for prob in subset:
            gt  = extract_boxed(prob.get("solution","")) or prob.get("solution","")
            r   = solve(prob.get("problem",""), budget=budget)
            pred    = extract_boxed(r["answer"]) or extract_boxed(r["thinking"])
            correct = answers_match(pred, gt)
            rfcs    = compute_rfcs(r["thinking"], gt) if correct else None
            accs.append(int(correct))
            tok_lens.append(r["thinking_len"])
            if rfcs is not None:
                rfcs_vals.append(rfcs)
        results.append({
            "budget":           budget,
            "accuracy":         round(statistics.mean(accs)*100, 1),
            "avg_rfcs":         round(statistics.mean(rfcs_vals),3) if rfcs_vals else None,
            "avg_thinking_len": round(statistics.mean(tok_lens)),
        })
    return results

10. Limitations and Discussion

Scale

The paper's main results use MATH-500 (500 problems) with multiple model families and multiple runs. This reproduction uses 50 problems and one model family. Statistical power is limited: confidence intervals on all percentages are wide. The novel findings (N1, N2, N3) are presented as preliminary signals, not definitive results.

Model Substitution

Claude Haiku with extended thinking and DeepSeek-R1-Distill-7B are different model architectures trained on different corpora with different RLHF procedures. A finding that differs from the paper may reflect a genuine behavioral difference between models rather than a flaw in the paper's claims. We cannot disentangle these without access to the original models.

SAGE Replication

SAGE relies on access to per-token log-probabilities during decoding, which is not available through the Anthropic Messages API. We cannot directly replicate the SAGE decoding algorithm. We instead reproduce the behavioral claims (RFCS, length-accuracy relationship, pass@k) and use the thinking budget ablation as a proxy for the compute-accuracy trade-off that SAGE optimises.

Answer Normalisation

Our normalisation is conservative. Cases like "1/2" and "0.5" are not matched even though they are mathematically equivalent. This may slightly undercount correct responses, making our accuracy estimates lower bounds.

Significance of the Novel Findings

The step classifier uses the same Claude Haiku model that produced the thinking traces. There is a risk that the classifier's categories reflect its own reasoning style rather than objective step types. The difficulty-RFCS correlation is computed across only the problems in our sample; many difficulty levels may have fewer than five data points. These limitations make the novel findings exploratory rather than confirmatory.

11. References

1. Zixuan Huang et al. "Does Your Reasoning Model Implicitly Know When to Stop Thinking?" arXiv:2602.08354, 2026.
2. Dan Hendrycks et al. "Measuring Mathematical Problem Solving With the MATH Dataset." arXiv:2103.03874, NeurIPS 2021.
3. Hunter Lightman et al. "Let's Verify Step by Step." arXiv:2305.20050, ICLR 2024.
4. DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." arXiv:2501.12948, 2025.
5. Chen et al. "Think Less, Achieve More: Cut Redundancy, Not Quality." 2025. (cited in paper for step segmentation)
6. Anthropic. "Claude Haiku 4.5." Model card and API documentation. 2025.