Evaluating Reasoning

A correct answer is useful evidence, but it leaves a question open: does the model succeed when the problem changes in a way that requires the same underlying method? One answer could have been recalled, guessed, or reached through invalid work that happened to end in the right place.

Reasoning evaluations approach that question through sets of problems, controlled variations, executable checks, and sometimes judgments of intermediate work. None gives a direct view of a model’s internal computation.

This follows contamination and eval harnesses. Those concerns still apply: hard questions can leak, and a parser or inference budget can change a reported result.

Multiple choice can still be difficult

Four answer choices provide information that an open response does not. Uniform random guessing gives 25% expected accuracy, and eliminating distractors can improve that rate. That is a baseline, not a floor: a model can systematically select wrong options and score below it.

But multiple-choice format does not make a task merely a test of recall. GPQA contains expert-written questions in biology, chemistry, and physics, with four answer options. Its difficulty comes from the content and distractors, not from asking the model to generate an unrestricted answer.

The original GPQA study reported 65% accuracy for domain experts, rising to 74% after discounting clear mistakes identified retrospectively. Skilled nonexperts averaged 34% despite access to the web. Use these numbers as comparisons within the original study’s human protocols.

Open-ended output removes the supplied candidates but introduces other choices: acceptable equivalent answers, numeric tolerance, parsing, and how to grade explanations. It can also be memorized.

What a score means mathematically

For a fixed evaluation set with answer keys $y_i$, an answer-only accuracy is

The judge may be an exact matcher, an execution test, or another evaluator. This expression records its decisions; it does not identify whether a successful answer came from recall, a learned algorithm, or a mixture.

A toy mixture makes the ambiguity explicit. If a system solves a fraction $r$ of questions and guesses uniformly among four options on the rest, its expected accuracy is $r+(1-r)/4$. That formula follows from the assumed two-path process. An observed accuracy does not prove that a real model actually uses those paths.

Even probabilities over choices need care. If you normalize likelihoods over the four offered candidates, the result is a distribution conditional on those candidates. It is not necessarily the probability that unconstrained generation would produce each answer, and it is not automatically calibrated to correctness.

Choosing tasks and reporting versions

FrontierMath uses expert-written advanced mathematics problems with answers designed for automatic checking. That supports reproducible outcome evaluation, but an accepted final answer does not certify every step of a supplied proof. Report the benchmark version and subset; performance claims age as models and evaluation protocols change.

ARC-AGI asks a system to infer transformations from examples and produce an output grid. Different benchmark generations and public or private splits are distinct evaluations. The allowed inference budget, number of attempts, and access to public tasks matter. Public grid tasks can be memorized or used for development like other public data.

Fresh tasks, controlled variations, and held-out problem families can strengthen the evidence for transfer. They still need quality checks. A rewritten question can accidentally change its answer, introduce ambiguity, or alter difficulty in ways unrelated to the ability being tested.

Inspecting the written work

A final-answer checker can accept a solution containing an invalid derivation. Conversely, a model can produce useful valid steps and then make an arithmetic error at the end. Step-level annotation makes those failures visible.

A process reward model learns to score intermediate steps from supervision on the process. An outcome reward model learns from labels on whole solutions. An outcome model may still read the entire solution; what differs is the supervision, not necessarily which text it can access.

Lightman et al.’s Let’s Verify Step by Step compared these approaches on MATH. Their process model scored each step and ranked complete solutions by multiplying those scores, counting neutral labels with positive ones in the reported method. A mean is another possible aggregation, but it can hide a single serious error among many routine steps. Neither rule automatically gives a calibrated probability that the full solution is correct.

A step scorer also makes mistakes. It may accept a plausible false claim, miss a dependency on an earlier error, or react to superficial formatting. Evaluate it against held-out annotations and failure cases. A readable chain of thought can be checked as text without assuming it faithfully reports the internal computation that produced the answer.

A worked comparison

For $2x+6=18$, the answer is $x=6$. Compare these two submitted derivations:

1. Subtract 6 to get $2x=12$, then divide by 2 to get $x=6$.
2. Subtract 6 to get $2x=12$, divide by 3 to get $x=4$, then state “therefore $x=6$.”

An answer-only matcher accepts both final values. A step review rejects the second derivation at the division step. The following code keeps manually supplied judgments of steps separate from the final-answer match.

The manual labels are doing the substantive verification. Replacing them with a learned scorer would introduce another model whose errors need measurement. Replacing them with a formal checker would require expressing the steps in a language that checker understands.

Test-time computation is part of the system

Sampling several candidates and selecting one can improve results under some conditions. Selection might use majority voting, an executable verifier, an outcome scorer, or a process scorer.

Report the candidate budget and selection procedure. “At least one candidate passed” assumes an oracle can identify that candidate; it differs from the success rate of the answer the system actually returns. Also track tokens, tool calls, elapsed time, and any task-specific retries needed to obtain the result.

When a test has little headroom

A high-scoring benchmark can still expose important failures and regressions. As the number of remaining errors shrinks, however, a small test has less resolution for comparing improvements. Repeated optimization against it can also narrow what the score represents.

Use paired item-level comparisons and keep regression tests. Add independently held-out tasks when you need evidence about abilities outside the old test’s coverage. A new hard benchmark is useful if its construction and scoring support the intended claim, not simply because the first published scores are low.

The tool-agent evaluation post extends this concern to systems whose answers include actions and changes to an environment.

Additional reading

• Rein et al., GPQA. Task format and the original expert and nonexpert studies.
• Glazer et al., FrontierMath. Advanced mathematics tasks and automated answer verification.
• ARC Prize, What is ARC-AGI?. Task design, benchmark generations, and evaluation splits.
• Lightman et al., Let’s Verify Step by Step. Process versus outcome supervision and whole-solution ranking.