Mercor attributes a 100% accounting score to Claude Opus 5: what it measures—and what it does not prove

The headline figure and what we can confirm

A result attributed to Mercor puts Claude Opus 5 at 100% on an accounting-task assessment. The available, not independently verified description characterizes the test as an evaluation of accounting work resembling month-end close activities and compares that score with 37% for human participants. Mercor’s post, meanwhile, is titled to say that AI now outperforms junior accountants. These details identify the claim and its general context, but they are not enough to reconstruct how the result was obtained.

That distinction matters: a striking percentage does not, by itself, tell us what counted as success, how many tasks were assessed, or what margin of error was accepted. The verifiable information available for this article does not include the number of exercises, the full protocol, enough task examples, a detailed scoring rubric, or the records needed to recalculate the score. It also does not establish whether 100% means every task was passed, an aggregate score reached that value, or some other metric was used.

Anthropic has an official page for Claude Opus 5 and platform documentation for the model. Those sources can help identify the model and consult official information about it, but the material available here does not confirm that Anthropic validated Mercor’s assessment. The result should therefore be attributed to Mercor, not presented as an independent certification by the model’s provider.

What is known about the test—and what remains unclear

The available description points to accounting tasks resembling those involved in a month-end close. That places the assessment in a recognizable professional context, but the phrase “similar to month-end close” does not identify the specific tasks. Based on the sources provided, we do not know whether participants were asked to classify transactions, reconcile records, review journal entries, prepare reports, or complete some other combination of activities. We cannot assign a specific task list to the benchmark without consulting and verifying its protocol.

The sample size and the way the exercises were constructed are also undocumented here. Those details are essential when interpreting a perfect result: completing a small set of cases correctly is not the same evidence as passing a large, diverse battery of tasks repeatedly. Without knowing how the tasks were selected and how they were scored, it is also impossible to assess whether the test favored a particular kind of response or covered unusual situations.

The comparison with people needs the same level of detail. A 37% figure has been reported, but the material available does not specify who the participants were, how many there were, what experience they had, or whether they faced exactly the same tasks under equivalent conditions. Mercor’s post title refers to junior accountants, but a title is not a substitute for a description of the human group or the procedure used.

What 100% means—and what it does not

Strictly speaking, a 100% result can be interpreted only in light of the metric’s definition and the set of tasks to which it was applied. If it represents the share of tasks passed, it would describe performance on those particular tasks. If it is a composite score, we would need to know how its components were combined. Since that definition is not included in the verified information accompanying this article, it is best to retain the attributed wording: Mercor reported a 100% result on its assessment.

The figure also does not, on its own, reveal whether the test measured serious errors, minor errors, incomplete responses, or abstentions. A system that recognizes it cannot resolve a case may behave more safely than one that answers confidently and gets it wrong. But comparing the two would require knowing how the benchmark treated abstentions and different types of failure. The information available here is not sufficient to say how those behaviors were scored.

A hypothetical example helps illustrate the distinction without attributing it to Mercor’s test. If an assessment asked a system to process ten routine documents and it handled all ten according to a fixed rubric, the result would describe that set. It would not automatically show that the system could handle incomplete documents, different accounting policies, regulatory changes, or discrepancies between records. Broader conclusions would require tests designed specifically for those situations.

An assessment is not the same as a real accounting close

A month-end close is a process situated within an organization. It involves working with information from different sources, applying internal policies and controls, resolving discrepancies, and leaving an audit trail that other people can review. This general description does not mean that every company follows the same procedure. That variation is precisely why a bounded task assessment is representative of real work only if it explains which parts it reproduces, what information it provides, and what it leaves out.

The available sources do not detail whether the model used external tools, how much time it had, how many attempts it could make, or what configuration it received. Those conditions can change what a comparison means. We also do not know whether the work was scored automatically, reviewed by experts, or assessed using a combination of methods. Without these details, it is not possible to judge reproducibility or rule out the possibility that evaluation decisions affected the score.

Comparisons with professionals also call for caution. A useful comparison should specify participants’ experience, give them the same instructions and materials as the model, set comparable time limits, and explain which tools were allowed. If the conditions are not equivalent, the difference between the percentages could reflect the design of the exercise as well as the capabilities of each group. The information provided does not let us confirm that the conditions were equivalent.

What would be needed to assess the claim

An independent replication would help establish whether the result holds outside the environment of the people who designed the assessment. To be informative, it should publish or describe the tasks, instructions, rubric, sample size, and scoring method in sufficient detail. It would also help to separate results by task category and explain failed, partial, and abstaining responses rather than provide only one overall percentage.

The test should identify the model and its configuration, as well as document the available tools, execution budget, and number of attempts. The human comparison should describe its participants and use reasonably equivalent conditions. The point is not to assume that every assessment must reproduce a complete workplace environment, but to state clearly which question it answers and which questions it does not.

If those details are published, readers and specialists could distinguish an improvement on bounded tasks from a broader ability to support accounting processes. Until then, the confirmed claim is narrower: Mercor reported a 100% score for Claude Opus 5 on an assessment described as similar to month-end close tasks, and a reported comparison puts human participants at 37%. The available verified material does not allow us to audit the calculation or conclude that the model has outperformed accountants under general workplace conditions.

Keep exploring

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.