The Aggregate
Unified LLM rankings, daily.
A public leaderboard I design, build and run on my own. It collects published results from hundreds of leaderboards and papers, puts them on one scale, and republishes a ranking, with uncertainty, every day.
- 1,600+ranked models
- ~25,000benchmark series
- 660+sources
- Pipeline
- Scrapers over public leaderboards and paper tables, refreshed daily. Automated audits gate every publish.
- Method
- A hierarchical IRT model puts heterogeneous benchmarks on one scale. Evidence rules discount vendor-selected results, and the fit is validated on held-out data.
- Product
- Model pages, comparisons, task-based shortlists, an MCP server, and a monthly forecasting duel against frontier LLMs.
- Python
- TypeScript
- Preact
- Cloudflare Workers
- R2
- GitHub Actions
- Claude Code