Artificial Analysis released Intelligence Index v5 on October 27, 2026, moving 60% of the composite to evaluations with private test sets or held-out answers. The update adds agentic coding and scientific research tasks, revises three existing evaluations, and changes the basis on which models compete for the top ranking.
The evaluator also launched Progress Index v1, a separate measure designed to compare model capability over time, even as the underlying benchmarks change. That distinction matters: the Intelligence Index evaluates models against today’s selected workload, while the Progress Index estimates how capability has developed across successive generations of tests.
In the launch results, Claude Opus 5.5 at maximum effort leads with an Intelligence Index score of 58, followed by GPT-6 Astra at 57 and GPT-6.1 Sol at 55. Those are Artificial Analysis’ measurements under its evaluation setup, not universal rankings for every application.
What Changed in Intelligence Index v5
The new index contains nine evaluations, grouped into four equally weighted categories. Each category contributes 25% of the composite, but individual benchmarks carry different weights.
| Category | Composite Weight | Evaluations and Their Weights |
|---|---|---|
| Knowledge work | 25% | AA-Briefcase, 15%; AutomationBench-AA, 5%; GDP.pdf, 5% |
| Coding | 25% | FrontierCode 2.0, 15%; Terminal-Bench 4.0, 10% |
| Science | 25% | Terminal-Bench-Science, 15%; CritPt v2 hard subset, 10% |
| General | 25% | AA-Omniscience v1.1, 15%; HLE-Diamond, 10% |
Within AA-Omniscience’s allocation, accuracy contributes 10 percentage points of the overall index and non-hallucination contributes five. Reliable handling of knowledge therefore affects the composite through both correct answers and the benchmark’s non-hallucination measure.
Artificial Analysis retains AA-Briefcase, AutomationBench-AA, GDP.pdf, and Terminal-Bench 4.0. It removes GDPval-AA v2.1, SciCode, and AA-LCR v1.1. The announcement provides the full replacement list and weights.
These are substantial changes to the measuring instrument. The two newly added benchmarks alone account for 30% of the composite. A model’s position can change because its strengths fit the new workload differently, even without a new model release.
Coding and Science Gain New Agentic Tasks
FrontierCode 2.0, developed by Cognition, contains 132 tasks in real open-source repositories. It grades changes against whether a repository maintainer would merge them, with a zero score for an attempt that fails any blocking check.
Artificial Analysis says it runs every task three times in Stirrup, its agentic harness. The tasks remain private to reduce contamination risk. This pushes the coding evaluation toward completing repository-level work under a merge-readiness standard.
Terminal-Bench-Science adds 70 scientific research tasks performed in a terminal environment. Artificial Analysis describes the tasks as expert-curated and reviewed. A task passes only when all its tests pass.
That all-tests requirement is consequential. An agent may make useful intermediate progress but still fail the task if its final work misses a required check. Buyers should read these scores as success under a specified completion standard, not a measure of how much partial assistance a model could provide.
Three Existing Evaluations Get Harder or Cleaner
The upgraded evaluations change both the questions and the conditions under which models answer them:
- HLE-Diamond replaces Humanity’s Last Exam with a refined, 1,000-question version, divided evenly between reasoning and knowledge.
- AA-Omniscience v1.1 removes saturated and public-set questions, adds new questions, and revises ambiguous or erroneous ones.
- CritPt v2’s hard subset contains 40 private research-level physics questions. Models receive an offline Python sandbox through Stirrup.
For CritPt, Artificial Analysis starts with 70 questions and excludes 30 that seven frontier models solved in all five attempts at launch. The resulting score focuses on the remaining difficult questions. Removing already-solved items helps preserve a benchmark’s ability to distinguish stronger models, but it also changes what the score represents.
Why the 60% Private-Data Shift Matters
Public benchmarks face a persistent measurement problem: their questions, answers, or worked solutions can enter training material or become targets for benchmark-specific optimization. A high score can then reflect a mixture of general capability, exposure, and preparation for that particular test.
Private tasks reduce opportunities for direct exposure. Held-out answers protect another part of an evaluation by withholding reference material. Artificial Analysis groups both approaches within its stated 60% figure, so the headline should not be interpreted as saying that 60% of every benchmark is completely secret.
It is also a share of the weighted composite, not necessarily 60% of all individual questions or task attempts. FrontierCode’s 132 tasks and CritPt’s 40-question subset illustrate why task counts and index weights are different quantities.
I’d give the shift more weight than a small movement at the top of the leaderboard. A test that remains informative after models improve is more useful for purchasing decisions than one on which leading systems have largely exhausted the available points.
Private data still brings a transparency tradeoff. Outside researchers cannot inspect undisclosed material as fully as an open benchmark, and secrecy alone cannot prove that no relevant exposure occurred. The available evidence here is Artificial Analysis’ published account; it does not independently verify the private sets’ integrity or reproduce the reported results.
The practical benefit is narrower but meaningful: v5 reduces reliance on openly available test material while refreshing questions that have become easier or less useful. It does not make contamination risk disappear.
The Progress Index Answers a Different Question
Artificial Analysis describes the Intelligence Index as a composite reflecting real-world usage today. Its benchmark mix can change as models improve and new tasks become relevant.
That creates a problem for historical comparisons. A score earned against an older, easier collection of tests is not automatically comparable with a score earned against a newer, harder collection.
Progress Index v1 uses item response theory to address that problem. According to Artificial Analysis’ published explanation, it estimates model capability and task difficulty together from benchmark results:
- Task difficulty is estimated from which models succeed or fail.
- Model capability is estimated from performance across tasks.
- Models evaluated on both older and newer benchmarks help connect those benchmark sets.
Imagine an older model evaluated on one group of tests and a newer model evaluated on a harder group. Comparing their raw results could understate the newer model’s improvement. Models tested on both groups provide a bridge for estimating the difficulty difference.
The Progress Index uses those relationships to put capability estimates on one continuous scale. It is a statistical estimate based on observed performance, not a retrospective rerun of every historical model on every current test.
The two indexes therefore serve different decisions:
| Reader’s Question | More Relevant Measure |
|---|---|
| Which evaluated models perform best on Artificial Analysis’ current task mix? | Intelligence Index v5 |
| How has estimated model capability changed across benchmark generations? | Progress Index v1 |
Artificial Analysis’ trend charts show each lab’s highest-scoring model available at a given time, using that model’s highest-scoring effort variant. Those lines describe a capability frontier. They do not describe the average model a lab offers or the performance buyers should expect at a fixed budget.
My view is that the Progress Index is the more appropriate starting point for claims about the pace of AI progress. The Intelligence Index remains more directly useful for building a current-model shortlist, provided its workload resembles the buyer’s needs.
What the Rankings Reveal Beyond First Place
Frequently Asked Questions
3 questions
1Does 60% Mean Most Individual Test Questions Are Private?
No. Artificial Analysis says 60% of the weighted composite uses private test sets or held-out answers. That is different from counting individual questions.
