Explainer
AI benchmarks explained: how do you tell which AI is best?
There’s no single “IQ test” for AI. These are the scoreboards that matter, what each one actually measures, and how we combine them into the AI Race index.
LMArena (formerly Chatbot Arena)
People ask a question, get answers from two anonymous models, and vote for the better one. Millions of votes turn into an Elo-style score, like chess ratings. It’s the best measure of how good a model feels to use, but it rewards style and confident answers, and gaps under about 5 points are noise.
Artificial Analysis Intelligence Index
An average over a set of hard tests in reasoning, maths, coding and knowledge, run independently by Artificial Analysis. It measures raw problem-solving rather than taste, and the test set is updated over time, so scores from different versions aren’t comparable.
Why one benchmark isn’t enough
- Labs tune for benchmarks. A model can top a test without being better in daily use.
- Benchmarks get stale. Once models ace a test, it stops telling them apart.
- Winning the race is more than one model. Release pace, attention and money decide who stays ahead.
How the AI Race index combines them
| Component | Weight | Built from |
|---|---|---|
| Capability | 55% | Average of Artificial Analysis and LMArena for each lab’s best model |
| Release velocity | 25% | Model launches on OpenRouter in the last 120 days (same-day listings count once) |
| Compute & capital | 20% | Valuation or parent market cap, log scale |
| Attention | 0% | News stories and Hacker News points: shown for context, never counted |
The headline number is the average of hourly readings over the last 7 days.
Right now Meta leads the index. See the model leaderboard, the best model right now or the live race.