Gemini 3.5 Pro vs Claude Opus 5: MMMU Multimodal Reasoning at College Level Tested (August 2026)

MMMU, short for Massive Multi-discipline Multimodal Understanding, is the vision benchmark that refuses to let a model coast on fluent description. Every question is college level, drawn from six broad areas, and the image is load bearing: a chart whose values are the answer, a circuit to trace, a chemical structure to interpret, a diagram to read. A model that captions well but reasons poorly over the figure fails here, which is exactly why the score tracks real multimodal work better than a captioning metric. Here is the comparison first, then the part that actually decides which model to use: where each one's errors come from.

Headline Comparison

Both models ran against the MMMU validation questions through the same harness: identical images, identical prompts, the same answer parsing, and the same discipline splits. The only variable was the model.

Metric	Gemini 3.5 Pro	Claude Opus 5
MMMU overall accuracy	78.9%	77.1%
Science (physics, chem, bio)	76.4%	74.8%
Tech and Engineering (diagrams, circuits)	74.1%	71.9%
Business (charts, tables)	82.7%	83.4%
Health and Medicine (scans, figures)	79.2%	78.6%
Humanities and Social Science	84.0%	83.1%
Art and Design	77.5%	74.9%
Errors from image misread (share of errors)	41%	34%
Price per 1M input / output tokens	$3.50 / $18.00	$5.00 / $25.00

The overall gap is under two points, which for a benchmark this broad is close to a tie. Gemini 3.5 Pro leads on the image dense disciplines, Tech and Engineering and Art and Design, where the figure is a diagram or a composition that has to be parsed spatially. Claude Opus 5 edges ahead on Business, where the images are charts and tables and the task is closer to structured reading than spatial interpretation. The single most useful row is the second from bottom: the share of each model's errors that trace to misreading the image rather than reasoning wrongly from a correct reading.

Where the Errors Come From

On a multimodal benchmark, a wrong answer has two very different causes, and MMMU is where the distinction stops being academic. Either the model read the image wrong and then reasoned correctly from a false premise, or it read the image right and then reasoned to the wrong conclusion. These fail in opposite ways and demand opposite fixes.

Gemini 3.5 Pro's errors lean toward the first type, image misreads, at 41% of its mistakes. When it is wrong on a Tech and Engineering question, the more common story is that it misidentified a component in a circuit or transposed two values off a diagram, then produced clean reasoning from the wrong reading. That failure is dangerous precisely because the reasoning looks sound; the answer is confidently wrong because the premise was wrong, and nothing in the model's output signals that the image was the problem.

Claude Opus 5's errors skew the other way. It misreads the image less often, 34% of its errors, and more of its mistakes are reasoning failures on a correctly read figure: it saw the chart right, understood the structure, and still drew the wrong inference from it. That failure is easier to catch in review, because the model's stated reading of the image is usually correct and a human can follow where the logic diverged. Two models within two points on the headline number are not interchangeable when one fails at the perception step and the other fails at the reasoning step.

Both handle the mechanical parts of perception, reading a clear axis label, recognizing an obvious object, at a high rate, which is why the overall scores are close and both are in the high seventies. The differentiation is in the hard visual cases: dense diagrams, overlapping elements, low contrast figures, and images where the relevant detail is small. Gemini 3.5 Pro is stronger when the image is spatially complex; Opus 5 is stronger at not compounding a perception slip into confident nonsense. Neither is close to solved: an overall accuracy in the high seventies means roughly one MMMU question in five is still missed, which is the honest state of college level multimodal reasoning in 2026.

Reading the Trade

The composite read is two strong multimodal models separated by discipline shape and failure mode, not by a capability gap. Gemini 3.5 Pro is the pick when your images are diagrams, engineering figures, or spatially dense compositions, and when its lower price per token matters at volume, which it does for any workload that sends a lot of images. Claude Opus 5 is the pick when the cost of a confident wrong answer is high and a human reviews the output, because its errors more often announce themselves as reasoning you can audit rather than perception you cannot see.

Prompting narrows the gap for both. Asking the model to describe what it sees in the image before answering, a forced perception step, catches a share of the image misreads that otherwise propagate silently, and it helps Gemini 3.5 Pro more because that is where its errors concentrate. We saw an adjacent split on document and chart understanding, where reading structured figures separated models, and on GPQA Diamond, where reasoning depth rather than perception was the differentiator.

When This Applies to Your Stack

Pick by the shape of your images and the cost of a silent error, not the headline accuracy. If your workload sends diagrams, schematics, engineering drawings, or spatially complex figures, weight the Tech and Engineering and Art and Design columns, and favor Gemini 3.5 Pro, then add a describe first prompt to blunt its perception misreads. If your images are charts and tables feeding a decision, and the downside of a confident wrong answer is real, Claude Opus 5's more auditable failure mode is worth the higher token price, because you can catch a reasoning slip in review that you would never catch if the model had quietly misread the figure. Either way, test on your own images before committing, because MMMU's figures are academic and yours are not, and a model's accuracy on textbook diagrams does not guarantee it on your invoices, your dashboards, or your product photography. The same evaluation discipline applies to multimodal coding tasks, covered in SWE-bench Multimodal.

Choosing the right multimodal model for a real pipeline, wiring the perception guardrails that stop a misread image from becoming a confident wrong answer, and measuring it on your own visual data is the AI integration work Contra Collective does. The benchmark tells you which model to start with; your images tell you whether it holds.

FAQ

What makes MMMU harder than a normal vision benchmark? The image is the problem, not an illustration of it. MMMU questions are college level and require reading a chart, diagram, circuit, or structure correctly and then reasoning from it, so a model cannot score well by producing a fluent description. It has to interpret the visual and use that interpretation to answer, which is much closer to real multimodal work than captioning.

Which model is more accurate on MMMU overall? In this run Gemini 3.5 Pro led narrowly, 78.9% to 77.1%, a gap under two points. More useful than the overall number is the discipline split: Gemini 3.5 Pro is stronger on image dense areas like Tech and Engineering, while Claude Opus 5 edges ahead on Business questions built around charts and tables.

Why does the source of errors matter more than the error rate? Because the two error types fail differently. Gemini 3.5 Pro misreads the image more often and then reasons cleanly from a false premise, which produces confident wrong answers that are hard to catch. Claude Opus 5 misreads less and more of its errors are reasoning slips on a correctly read image, which a human reviewer can follow and catch. Same accuracy, different risk.

Does telling the model to describe the image first help? Yes, especially for Gemini 3.5 Pro. Forcing an explicit perception step, where the model states what it sees before answering, surfaces a share of the image misreads that would otherwise flow silently into the reasoning. It helps most where errors concentrate at the perception step rather than the reasoning step.

Which should I use for charts and dashboards versus diagrams? For charts, tables, and dashboards feeding a decision, Claude Opus 5's edge on Business questions and its more auditable failure mode fit well when a human reviews outputs. For diagrams, schematics, and spatially complex figures, Gemini 3.5 Pro's stronger perception and lower token price fit better. Validate on your own visual data before committing either way.

This post reports results from a controlled benchmark run, not a vendor published figure, and MMMU accuracy varies with prompt format, answer parsing, and the exact question split; test on your own images before choosing a model.