MKA evaluates AI agents on multi-step professional tasks that require them to interpret information across visually complex files and produce a finished deliverable. Google DeepMind announced the benchmark with Artificial Analysis on November 10, 2026. In the report’s four-model comparison, Gemini 4 Carbon placed second on the main score, but the top three models were statistically indistinguishable on that measure. (Google DeepMind’s announcement; MKA technical report)

A leaderboard position is an ordering of reported scores; the statistical analysis determines whether the gap between them is clear.

MKA tests finished work across complex files

The Multimodal Knowledge Agent benchmark is designed for professional work that draws on multiple kinds of visual material—not just a question about a single image. A task might involve interpreting drawings, charts, schematics or maps, then creating a usable output. The private evaluation has 200 tasks across 10 sectors and 83 occupations. Each task uses between 2 and 45 files; 176 tasks require the agent to create or modify a standalone file. (MKA technical report)

That makes the benchmark’s focus the whole job: gathering relevant information from a set of inputs and producing work that meets task-specific criteria. A separate public showcase has 10 tasks, distinct from the 200-task private evaluation. (MKA technical report)

What Climb-WSR measures

Climb-WSR is the average, across tasks, of each model’s weighted share of satisfied criteria in the harder part of the scoring rubric. It is not the percentage of tasks the model completed in full. A score of 54.34% for Gemini 4 Carbon means it met that share of the weighted Climb criteria on average across the evaluated tasks. (MKA technical report)

The report also gives its automated grader a 96.2% pairwise agreement rate with human raters. That figure comes from comparing judgments on criteria across 50 tasks in nine sectors; it measures agreement, not overall benchmark accuracy. The study reports a 94.7–97.6% confidence interval for agreement and a weighted Cohen’s κ of 0.911. (MKA technical report)

The scores rank models, but the top three are statistically indistinguishable

Google DeepMind’s technical report lists four models’ results under the same MKA evaluation setup:

ModelClimb-WSROverall WSRMean turns per taskMean API cost per task
Claude Fable 5.256.56%73.26%128.8$35.28
Gemini 4 Carbon54.34%71.89%46.6$5.63
Claude Opus 5.553.01%70.98%91.4$10.29
GPT-6 Astra50.24%69.34%29.0$6.03

The reported Climb-WSR order puts Claude Fable 5.2 first and Gemini 4 Carbon second. Their difference is 2.22 percentage points, with a paired 95% confidence interval from −0.94 to +5.49 points. Because that interval crosses zero, the report’s analysis does not distinguish their scores statistically; it likewise finds the top three statistically indistinguishable on Climb-WSR. The ranking alone therefore does not establish a clear quality gap among those three models. (MKA technical report)

The cost figures are average API costs per task at provider list prices, including prompt-cache read and write pricing. On those terms, the report gives Gemini 4 Carbon a mean cost of $5.63 per task and Claude Fable 5.2 a mean of $35.28. Cost is a separate result from the quality score, and both figures are specific to the benchmark’s workload and stated pricing assumptions. (MKA technical report)

Removing image viewing lowered Carbon’s score

In a separate experiment, the report removed the view_image tool for Gemini 4 Carbon while leaving tools such as code execution, web search, parsing, OCR and computer-vision libraries available. Carbon’s Climb-WSR fell by 5.94 percentage points, and its Overall WSR fell by 4.58 points. In that setup, the model also used 1.85 times as many input tokens, 1.40 times as many output tokens and 1.28 times as many turns as in the condition with image viewing. These results describe Carbon’s performance in this MKA experiment. (MKA technical report)

The four-model comparison ran each model once per task. Its reported ranking and averages describe those evaluation runs; they do not provide a repeated-run reliability estimate. (MKA technical report)