Artificial Analysis Shakes Up AI Benchmarks
The firm's overhauled testing system, with many tests now kept secret, widens the gap between open-weight Chinese AI models and the overall leader.
Updated on Nov. 3, 2026 in Artificial Intelligence
Live Poll
Do you trust large American AI companies to keep their promises to users?
Artificial Analysis, an independent firm that measures AI performance, has overhauled its testing system and kept many of its tests secret. Under the new system, the gap between the best open-weight AI models and the overall leader grew from 3 points to 12.
Why it matters
Keeping tests secret makes it harder for AI companies to train models specifically to score well on known benchmarks. The change also reshapes how open-weight models from China compare with leading American systems.
Under the new system, the best open-weight models scored 32 points while the overall leader, Anthropic's Claude Opus 5.5, scored 58 points. A 12-point gap now separates them, up from 3 points under the previous testing system.
The players
Artificial Analysis
An independent firm that measures and ranks the performance of AI models across a range of benchmarks.
Sridhar Ramaswamy
A technology executive who stated that the open-model gap widened from 3 points to 12 when the benchmarks went private.
Anthropic
An American AI company whose Claude Opus 5.5 model ranked as the overall leader on the new benchmarks.
The details
Artificial Analysis rebuilt its testing to better reflect how people actually use AI, and many tests are now private. Claude Haiku 5.5 outperformed every AI model with publicly released weights under the new rankings.
Timeline
Under the previous testing system, open-weight models trailed the overall leader by 3 points.
Business Insider published its article on the new rankings on November 3, 2026.
The Tech Race
The shift to private testing responds to benchmark hacking, the practice of tuning AI models to publicly known evaluations. The new rankings may also strengthen the investment case for Anthropic's planned IPO and a possible future OpenAI listing.
Readers comparing AI tools will find that rankings based on secret tests may differ from earlier public leaderboards. Choosing a model may require weighing published scores against independent evaluations that are harder to game.
The takeaway
Benchmark scores depend heavily on how they are tested, so looking at evaluation methods helps readers judge AI performance more accurately.
Further reading
Explore more on Artificial Intelligence to see how benchmark design shapes the race between leading AI models.
Source note: This article includes information reported by Business Insider.
Live Poll
Do you trust large American AI companies to keep their promises to users?
