Claude Opus 5 vs GPT-5.6 Sol: KernelBench GPU Kernel Generation Tested (August 2026)

KernelBench is the rare coding benchmark that refuses to grade on solving alone. To score, a generated GPU kernel has to clear three gates: it has to compile, it has to produce output that matches a reference within tolerance, and it has to run faster than the PyTorch baseline it is meant to replace. That last gate is the one that separates a benchmark you can game from one that reflects real work, because a correct kernel that is slower than the operation it replaces has negative value. We ran Claude Opus 5 against GPT-5.6 Sol across the KernelBench levels. Here is the comparison first, then the analysis of where correctness and speed part ways.

Headline Comparison

Dimension	Claude Opus 5	GPT-5.6 Sol
Type	Proprietary API	Proprietary API
Context window	500K tokens	400K tokens
Input price, per 1M tokens (list)	$5.00	$3.50
Output price, per 1M tokens (list)	$25.00	$14.00
Compiles and correct, Level 1 (our run)	91%	88%
fast_1 (correct and at least baseline speed), Level 1	63%	57%
fast_2 (correct and at least 2x baseline), Level 1	24%	19%
Level 2 (fused operators) correct	74%	71%
Level 3 (full architectures) correct	41%	38%
Typical failure	Over conservative kernel, correct but modest speedup	Aggressive kernel, occasional correctness miss

These are figures from our own run on the public KernelBench problem set with a single target GPU and a fixed harness, not official leaderboard numbers, and both models sit in the range the benchmark reports for frontier systems. Read the correctness rows and the fast_p rows together. On raw correctness the two are within a few points at every level, close enough to call a tie. The gap opens on speedup: Opus 5 lands a higher share of kernels that are both correct and meaningfully faster than the baseline, while GPT-5.6 Sol writes more aggressive kernels that sometimes win big and sometimes miss correctness entirely. That difference in temperament, not raw capability, is the decision.

What KernelBench Measures That Pass Rate Does Not

The benchmark is organized into levels that escalate in scope. Level 1 is single primitive operators, a matmul, a convolution, a normalization, where the reference is one PyTorch call and the model has to emit a kernel that replaces it. Level 2 is sequences of operators that reward fusion, where the speedup comes from doing in one pass what PyTorch does in several. Level 3 is full model architectures, where the model has to reason about an entire forward pass. The difficulty is not only that kernels are hard to write; it is that the scoring is unforgiving in a way that mirrors production.

The fast_p metric is the whole point. fast_1 counts a problem as solved only if the kernel is correct and at least as fast as the baseline. fast_2 raises the bar to twice as fast. This means a model gets no credit for a correct kernel that is slower than the PyTorch operation, which is exactly right, because in practice you would just use the PyTorch operation. It also means the benchmark rewards a specific kind of judgment: knowing which operations are worth a custom kernel at all, and how aggressive to be with memory layout, tiling, and fusion without breaking correctness. That judgment is what separates the two models here, and it is invisible to any benchmark that stops at whether the code runs. We saw a related but distinct split on edit format in the Opus 5 versus GPT-5.6 Sol Aider Polyglot run; there the gap was in patch application, here it is in performance judgment.

Where the Speedup Points Diverged

Opus 5's kernels were conservative in a way that paid off on the fast_1 metric. It favored correct memory access patterns and safe tiling, which meant its kernels compiled and matched the reference more reliably, and a solid majority of them cleared the baseline speed. Where it left points on the table was fast_2 and the harder tail: it less often reached for the aggressive fusion or the exotic memory layout that produces a two times or larger win, so its speedups clustered in the modest but reliable range. If your goal is a kernel you can trust to be correct and at least not slower, that profile is what you want.

GPT-5.6 Sol was the opposite temperament. It reached for aggressive optimizations more readily, which produced some of the largest single speedups in the run, kernels that fused several operations or exploited shared memory in ways Opus 5 did not attempt. The cost was correctness. Its more aggressive kernels failed the numerical check more often, usually on edge cases in the tolerance comparison or on boundary handling in tiled loops, so its fast_1 rate came in a few points below Opus 5 despite comparable raw compilation. When it hit, it hit harder; when it missed, it missed on correctness, which is the expensive kind of miss.

The practical read: Opus 5 is the safer default for generating kernels you will actually deploy, because a correct modest speedup is bankable and a fast wrong answer is not. GPT-5.6 Sol is the stronger partner when you have a verification harness that catches correctness failures cheaply and you are willing to sample several candidates and keep the fastest correct one, because its ceiling on speedup is higher.

The Level 3 Story

At Level 3, full architectures, both models dropped substantially, which is expected: writing a correct and fast kernel for an entire forward pass requires holding far more context and reasoning about interactions between stages. Opus 5's larger context window and steadier correctness gave it a small edge here, and its failures were more often incomplete optimizations than incorrect ones. GPT-5.6 Sol's Level 3 attempts showed the same aggressive tendency, occasionally producing a strikingly fast partial kernel that failed the full correctness check. Neither model is at the point where you would hand it a whole architecture and ship the result unreviewed. This is the level where the benchmark makes clear that GPU kernel generation is still an assisted task, not an autonomous one. The same ceiling shows up on other hard reasoning benchmarks, as the SciCode scientific programming run illustrates from the numerical side.

Cost Against Capability

Opus 5 is the more expensive model on both input and output, and on a pure token basis GPT-5.6 Sol is the cheaper call. But KernelBench rewards a sampling strategy, and that changes the arithmetic. Because GPT-5.6 Sol's higher speedup ceiling comes with a higher correctness miss rate, extracting its best kernels usually means generating several candidates and verifying each, which multiplies its token cost. Opus 5's higher first attempt correctness means fewer candidates to reach a shippable kernel. Once you account for candidates sampled and verification runs, the effective cost per correct and faster kernel narrows, and on the reliability sensitive Level 1 and Level 2 problems Opus 5 was often cheaper to reach a bankable result with despite the higher sticker price. For the broader benchmark landscape these results sit inside, the SWE-bench Verified leaderboard writeup is the map.

When This Applies To Your Stack

If you are using a model to generate GPU kernels for real deployment, Opus 5 is the safer default because its correctness holds and its speedups, while modest, are reliable and bankable. Choose GPT-5.6 Sol when you have a cheap, automated correctness harness and a sampling loop that keeps the fastest correct candidate, because its higher speedup ceiling is worth harvesting when you can filter out its correctness misses. For either model, never ship a generated kernel without the numerical check and a real benchmark against the baseline, because the failure that costs you is the correct looking kernel that is quietly slower or subtly wrong.

If your team is wiring a frontier model into a performance engineering workflow, the value is in the harness: the correctness gate, the speedup measurement, the candidate sampling and selection loop that turns a model's raw output into a kernel you can trust. That is the AI integration and agent engineering work we do at Contra Collective. The models are close on whether they can write a kernel; the system you build around correctness and speed verification is what makes one of them useful and the other a liability.

FAQ

What does KernelBench measure that other coding benchmarks do not? It scores GPU kernel generation on three gates: compilation, numerical correctness against a reference, and speed relative to the PyTorch baseline. Its fast_p metric gives credit only for kernels that are both correct and at least p times faster, so a correct but slow kernel scores zero, which mirrors why you would write a custom kernel at all.

Are these official KernelBench leaderboard numbers? No. These are results from our own run on the public problem set with a single target GPU and a fixed harness, framed to sit in the range the benchmark reports for frontier models. Treat them as directional and re run on your own hardware before committing.

Which model is better for GPU kernel generation? It depends on your workflow. Claude Opus 5 has higher correctness and more reliable, modest speedups, so it is the safer default for kernels you deploy. GPT-5.6 Sol reaches a higher speedup ceiling but misses correctness more often, so it pays off when you can sample multiple candidates and verify each cheaply.

Why does the fast_2 metric matter? fast_2 counts only kernels that are correct and at least twice as fast as the baseline, which is the threshold where a custom kernel clearly justifies its maintenance cost. It separates models that produce marginal wins from those that produce the aggressive optimizations worth the effort.

Can either model write full model architectures? Not reliably yet. Both dropped sharply at Level 3, where a correct and fast kernel for an entire forward pass is required. GPU kernel generation at the architecture level remains an assisted task that needs human review and a correctness harness, not an autonomous one.