Three Coding Agents, Two Numbers: How to Evaluate the Agentic Coding Market After Composer 2.5

The coding agent leaderboard looked simple three months ago. Two dominant tools at the frontier, a clear performance gap below them, and a cost structure that tracked performance more or less proportionally. That picture has changed.

Per Artificial Analysis benchmarks published May 21, Cursor Composer 2.5 reportedly scores 62 on the Coding Agent Index. Claude Opus 4.7 max reportedly scores 66. GPT-5.5 xhigh reasoning reportedly scores 65. Three tools in a four-point band. The cost structure for those same three tools spans a factor of 60.

That’s not a subtle development. It’s a structural change in how enterprise development teams should evaluate this market.

The price-performance map

The comparison is worth stating plainly before analyzing it. All figures attributed to Artificial Analysis, source URL pending pipeline resolution; treat as reported until confirmed.

Coding Agent Index scores (reported): Claude Opus 4.7 max at 66, GPT-5.5 xhigh reasoning at 65, Cursor Composer 2.5 at 62. Cost per task: $4.10 for Opus 4.7, $4.82 for GPT-5.5, $0.44 for Composer 2.5 in Fast mode, $0.07 in Standard mode.

The score gap between first and third is about 6%. The cost gap between first and third, at Composer’s Fast mode, is approximately 89%. At Standard mode, it’s 98%.

SWE-Bench-Pro-Hard-AA adds dimension. This benchmark tests software engineering work on harder variants of the SWE-Bench task set, bug fixing, feature implementation, working with real repository code. Per Artificial Analysis, Composer 2.5 reportedly moved from 12% to 47% on this benchmark, reportedly matching Opus 4.7 max. If that figure holds on independent verification, it means Composer 2.5 isn’t third on coding-specific tasks, it’s tied for the top of a relevant evaluation subset, at Standard pricing that’s 60x cheaper.

Don’t build a procurement decision on that yet. The SWE-Bench-Pro-Hard-AA score is reported from a single benchmark source with no URL confirmation at this stage. Verify before acting.

What Artificial Analysis measures, and what it doesn’t

Artificial Analysis is an independent benchmark provider, not a vendor. That matters for how much weight you give these numbers. Their evaluations cover real-world task performance on standardized coding benchmarks, not marketing-selected demos.

The Coding Agent Index methodology covers task completion on structured software engineering problems. What it doesn’t capture, by design: latency at production scale, behavior on your specific codebase, quality of generated code under review by your team’s standards, or cost under your actual usage patterns. An index score is a starting point for evaluation, not a replacement for it.

The part that’s missing from this brief: execution wall time. The Wire referenced 6.7 minutes average for Composer 2.5 in Fast mode, reportedly third-fastest on the index. That figure didn’t come with a source URL and doesn’t appear here. Speed matters for developer experience. Request it from the Artificial Analysis report when the URL resolves.

Cursor vs. Claude Code vs. Codex: the competitive positions

Each vendor is playing a different game. Understanding their positions helps interpret the benchmark picture.

Anthropic’s Claude Code and Codex (OpenAI) are flagship-tier products priced to match. They’re not trying to win on cost, they’re positioned as the highest-capability tier, and their pricing reflects that framing. For enterprise teams with compliance requirements, existing vendor contracts, or workflows deeply integrated with one ecosystem, the price premium may be rational.

Cursor is a third-party tool with no hyperscaler parent. Anysphere’s path to market share runs through price-performance disruption. Composer 2.5, if the benchmarks hold, is a serious execution of that strategy. Three months ago, a tool claiming third place on the Coding Agent Index would have been four to six points behind the leaders. Four points is a different conversation.

The context for this competitive shift: the May 21 brief on what Google, Anthropic, and OpenAI actually built in their agentic coding platforms mapped the three-lab picture before Composer 2.5’s entry. That brief is the baseline. Composer 2.5 inserts a non-lab competitor into a market the three major labs were treating as their to structure.

The pricing gap thesis

A brief published here on May 13, when frontier model benchmarks converge, pricing gaps become the story, made a specific argument: as the performance ceiling in a model category gets crowded, cost per token (or per task) becomes the primary competitive variable. Composer 2.5 is the clearest evidence yet for that thesis.

The May 13 brief identified the pattern in general LLM benchmarks. The Coding Agent Index is a narrower, task-specific evaluation, harder to game, more directly relevant to developer purchasing decisions. The convergence dynamic is more advanced here than it is in general LLM benchmarks, and the cost disparity is starker.

What Anthropic and OpenAI do with pricing in response to this is the variable to watch. A four-point benchmark advantage doesn’t sustain a 10x price premium indefinitely if enterprise procurement teams internalize the Artificial Analysis data. The historical pattern in infrastructure software markets: when a challenger demonstrates cost-competitive performance, incumbents either cut prices, differentiate on non-benchmark dimensions (support, integrations, compliance certifications), or both.

Enterprise decision framework

The evaluation question isn’t “is Composer 2.5 better?” It’s: does it perform well enough on the tasks your team actually runs, at a cost that justifies migration risk?

Three things to evaluate before switching:

First, verify the benchmark source. The Artificial Analysis report, once the URL resolves, is your primary document. Check the SWE-Bench-Pro-Hard-AA methodology, specifically, whether the task distribution matches your codebase’s profile. A 47% score on a benchmark set skewed toward Python bug fixes tells you less if your team writes Go infrastructure.

Second, run your own task sample. Index scores aggregate across a benchmark set. Your use cases are a subset. Run Composer 2.5 on a representative sample of your team’s actual tasks, ideally the same tasks you’d run against Opus 4.7 or GPT-5.5, and compare outputs at both models’ respective costs. The Standard pricing at $0.07 makes the evaluation cost negligible.

Third, account for switching costs. Integration, workflow changes, team retraining, and the time cost of a failed migration all factor into the real economics. A 10x cost reduction doesn’t justify a disruptive migration if the capability gap on your specific tasks is material.

What to watch

Artificial Analysis publishes Coding Agent Index updates regularly. If Composer 2.5’s score holds or improves in the next update cycle, the pressure on incumbent pricing increases. If it regresses, the May 21 numbers were a snapshot, not a trend.

TJS synthesis

The agentic coding market is repricing. Not spectacularly, four benchmark points isn’t a rout, but the cost structure has shifted in a way the major labs will have to address. For teams on Claude Code or Codex: run the evaluation at $0.07 per task. The cost of not knowing whether Composer 2.5 works for your use case is now essentially zero. The cost of staying on a 60x more expensive tool without checking is real.