Claude Code vs Codex: Terminal-Bench 4.0 Results Tested (September 2026)
Terminal-Bench 4.0 is a 66 task CLI benchmark spanning software, machine learning, science, hardware, security, media, and operations work, each task run five independent times with an eight hour agent timeout. In September 2026 the top two rows are Codex paired with GPT-6 Astra at 58.2 percent and Claude Code paired with Claude Fable 5.1 at 57.9 percent, a gap small enough to sit inside run to run variance. But the leaderboard also has Claude Code paired with Claude Opus 5 at 52.4 percent, six points below the Fable 5.1 pairing on the exact same agent. The model matters as much as the harness, and this post works through what that pairing effect means for how you should actually choose a coding agent stack.
Claude Code vs Codex: Terminal-Bench 4.0 Results Tested (September 2026)
Terminal-Bench 4.0 is the benchmark built to answer a question SWE-bench never could: can an agent actually get work done inside a terminal, across a task set wider than repository bug fixing. It covers 66 tasks spanning software engineering, machine learning, science, hardware, security, media, and general operations, and each task runs five full independent trials under an eight hour agent timeout, with the leaderboard reporting run to run variance rather than a single lucky number. In September 2026, the top of that leaderboard is a near tie: Codex paired with GPT-6 Astra at 58.2 percent, Claude Code paired with Claude Fable 5.1 close behind at 57.9 percent. The gap is inside the benchmark's own noise. What is not noise is a second row further down: Claude Code paired with Claude Opus 5 scores 52.4 percent, six full points under the same harness running a different model.
| Agent harness | Model | Terminal-Bench 4.0 score | Notes |
|---|---|---|---|
| Codex | GPT-6 Astra | 58.2% | Current leaderboard top |
| Claude Code | Claude Fable 5.1 | 57.9% | Within run to run variance of first place |
| Claude Code | Claude Opus 5 | 52.4% | Same harness, six points below the Fable 5.1 pairing |
| Codex | GPT-5.6 Sol | Lower overall, ties on business ops tasks | Cheaper per task, competitive on a specific task category |
Why the harness and the model are two separate variables
The number everyone quotes from a coding benchmark is usually attributed to the model. Terminal-Bench 4.0's leaderboard format, which scores a harness and model pairing as a single row rather than crediting the model alone, makes clear that this attribution is wrong often enough to matter. Claude Code running Claude Opus 5 and Claude Code running Claude Fable 5.1 are the same tool calling loop, the same terminal access, the same eight hour budget. The only variable that changes is the model behind it, and the score moves by six points. That is a larger swing than the entire gap separating first and second place on the leaderboard.
The practical implication is that a claim like "Claude Code scored 58 percent" is incomplete in a way that changes the decision it is meant to inform. The harness sets the ceiling for how well a model's capabilities translate into finished terminal work: how it retries, how it recovers from a failed command, how much context it keeps live across a long task. The model sets how far the agent gets when the harness gets out of its way. Comparing tools without pinning down which model sits behind each one is comparing incomplete pairs, which is exactly the trap we flagged from the model quality side in SWE-bench Verified and what the score actually measures: a single number, taken out of its harness context, tells you less than it appears to.
What the near tie at the top actually means
A 0.3 point gap between Codex plus GPT-6 Astra and Claude Code plus Claude Fable 5.1 is not a result you should plan a procurement decision around. Terminal-Bench 4.0's own methodology, five trials per task with reported variance, exists precisely because a single run of 66 tasks produces noise on that scale from randomness in agent behavior alone. Treat the top two rows as tied and look at the axes that actually differentiate your use case: task category performance, cost per completed task, and latency under your specific workload, not the headline percentage.
Where the leaderboard is informative is in the categories underneath the top line score. Terminal-Bench 4.0 breaks out performance across software, machine learning, science, hardware, security, media, and operations, and the pairings do not rank identically across every category. GPT-5.6 Sol on Codex, a cheaper model than GPT-6 Astra, ties on business operations tasks despite trailing on the aggregate score. If your actual workload skews toward one of these categories rather than an even mix, the aggregate ranking can point you at the wrong tool entirely. We covered the same category level nuance from the adoption side in the coding agent adoption and Terminal-Bench landscape, and the lesson repeats here: pull the category breakdown before you pull the headline number.
The composable stack changes what "choosing a tool" even means
Terminal-Bench 4.0's harness plus model format is also a reflection of how teams are actually deploying these agents in September 2026. Claude Code, Codex, and Cursor are increasingly run together rather than as exclusive choices, with orchestration, execution, and review split across the three rather than consolidated into one. We laid that shift out in Cursor, Claude Code, and Codex as a composable stack, and Terminal-Bench 4.0's scoring format is a quiet acknowledgment of the same reality: the question is not which single agent wins, it is which model you pair with which harness for which category of task. A team running a composable stack can route hardware or security category tasks to whichever pairing scores best in that specific column, rather than committing every task type to one tool's aggregate score.
Cost per completed task is the number the leaderboard does not show you
An eight hour agent timeout across five trials on 66 tasks is an expensive benchmark to run, and it is an expensive pattern to replicate in production if you are paying per token for a frontier model on every task regardless of difficulty. GPT-6 Astra and Claude Fable 5.1 are both premium tier models, and a pairing that trails by a point or two on the aggregate score but costs meaningfully less per completed task, such as the GPT-5.6 Sol pairing on business operations work, can be the better choice for high volume, lower stakes task categories even though it will never top the leaderboard. The methodology writeups from the benchmark maintainers report scores, not dollar cost per completed task, so that calculation is one you have to build yourself against your own task mix.
FAQ
Is Codex better than Claude Code based on Terminal-Bench 4.0? The top pairings are within 0.3 points of each other, inside the benchmark's own run to run variance. Treat the top of the leaderboard as a tie and look at category level breakdowns and cost per task for your specific workload instead of the aggregate score.
Why does the same harness score so differently with different models? Claude Code paired with Claude Opus 5 scores 52.4 percent against 57.9 percent for the same harness paired with Claude Fable 5.1. The harness sets the ceiling for how tool calling and recovery behavior translate into finished work, but the model determines how far the agent actually gets within that ceiling.
What does Terminal-Bench 4.0 actually test? 66 tasks across software engineering, machine learning, science, hardware, security, media, and general operations, each run five independent times under an eight hour agent timeout, with variance reported alongside the score.
Should I pick one agent for every task category? Not necessarily. Category level scores do not rank identically to the aggregate, and a composable stack that routes different task categories to different harness and model pairings can outperform committing everything to one tool's overall ranking.
When this applies to your stack
If you are choosing or standing up a coding agent stack, the aggregate Terminal-Bench 4.0 number is a starting point, not a decision. Pin down which model sits behind each harness you are evaluating, pull the category breakdown for the task types your team actually runs, and price out cost per completed task rather than cost per token. That evaluation work, plus the agent engineering to route tasks to the right pairing in production, is the kind of applied AI work we do at Contra Collective.
More from the lab.
Gap and Christie's Are Running Production AI on Mac Minis: What Apple's Q4 Call Actually Revealed About Enterprise Local Inference
On its fiscal Q4 earnings call on November 2, 2026, Apple named two enterprise customers doing something more specific than the usual AI PC talking point: Gap runs AI loss prevention models across hundreds of stores entirely on Mac minis, which CFO Kevan Parekh described specifically as eliminating cloud token costs, and Christie's runs pricing estimate models on Mac Studio. Apple's pitch is that a few thousand dollar Mac can produce inference value worth tens of thousands over its life. That math is directionally right for a narrow set of workloads, and it is not a general argument against cloud inference, even though the earnings call framing invites you to read it that way.
Kimi K3.1 Pro: Moonshot's Open Weight Model and What a 1M Token Context Actually Changes
Kimi K3.1 Pro went live in Moonshot's API on October 23, 2026, priced at 2 dollars per million input tokens and 10 dollars per million output tokens, with a 1 million token context window. Moonshot promised open weights by November 4. The model inherits Kimi K3's reported 2.8 trillion total parameters and 104 billion active parameters per token, which means the real question for most teams is not whether it is open but whether anything they own can actually run it.
oMLX on Apple Silicon: What the SSD KV Cache Actually Buys Coding Agents
oMLX search interest has climbed fast this year, and the pull is a specific engineering claim: a coding agent's time to first token dropping from 90 seconds to as little as 3 once a long prompt's key value cache is already sitting on SSD instead of being recomputed from scratch. The architecture is real and the restore speed is measurable. The claim is also softer than the headline number suggests, and this post works through both halves.