ExpertCrowd Benchmark Ranks Claude Opus 5.5 Top in Coding
The new ExpertCrowd-SWE test found a 35-point score gap among seven leading AI models on software engineering tasks.
Updated on Nov. 12, 2026 in Artificial Intelligence
ExpertCrowd-SWE, a benchmark built with ExpertCrowd, Toptal AI, and HUD, tested seven leading AI models on newly written software engineering tasks. Anthropic's Claude Opus 5.5 scored highest with a 67.3% average, while SpaceXSI's Grok 4.6 finished seventh at 32.2%.
Why it matters
The results show that models which appear evenly matched on the existing DeepSWE benchmark can differ widely on harder, newly written tasks. Developers and buyers comparing AI coding tools may find that older leaderboards do not reflect performance on less familiar work.
The benchmark drew on 87 open-source projects across 11 programming languages, with each model given a full code copy and a sandbox to read files, run commands, and edit code. Each task was attempted five times with advanced reasoning enabled and scored against an engineer-written rubric.
The players
Anthropic
Anthropic is an AI company that develops the Claude family of models, and its Claude Opus 5.5 model ranked first in this benchmark.
OpenAI
OpenAI is an AI research company known for the GPT series of models, and its GPT-6 Astra placed second in this benchmark.
Toptal AI
Toptal AI is part of Toptal, a freelance talent network that supplied the software engineers who built the benchmark tasks.
ExpertCrowd
ExpertCrowd is a platform that organizes expert contributors, and it co-produced the ExpertCrowd-SWE benchmark with Toptal AI and HUD.
HUD
HUD is an organization that partnered on the ExpertCrowd-SWE benchmark alongside ExpertCrowd and Toptal AI.
The details
Software engineers from Toptal's talent network wrote the tasks, choosing areas where models tend to struggle. Each model's five attempts per task were scored against a rubric written by the engineer who created that task.
Timeline
November 12, 2026: Article published reporting the ExpertCrowd-SWE benchmark results.
Roadmap
The ExpertCrowd-SWE results suggest that leaderboards built on DeepSWE may have masked real differences between frontier coding models. Testing on newly written tasks is emerging as a way to separate models that look equivalent on established benchmarks.
Developers choosing an AI coding assistant may find that a model's general benchmark ranking does not predict how it handles unfamiliar engineering tasks. Comparing performance on newly built tests can help match a tool to the specific kind of work a team does.
The takeaway
Benchmark rankings can shift sharply when tasks change, so checking a model against work similar to your own is more reliable than relying on a single leaderboard score.
Further reading
For more on how these systems are evaluated, see Artificial Intelligence coverage.
