OPEN-MODEL LEADERBOARD
Measured here.
Sourced elsewhere.
The open-weight models Undergrove runs, side by side: benchmark scores as their publishers report them, each linked to the page it came from, and speed we measure ourselves every week. Nothing here is estimated.
What each publisher reports.
Scores from each model’s own model card, as written there. Press a benchmark to rank by it; a dash means the publisher doesn’t report that benchmark for that model, so we don’t either.
Ranked by GPQA Diamond, highest first.
| Model | ||||||
|---|---|---|---|---|---|---|
| Kimi K3 | 93.5%max | 43.5%max · no tools | 56.0%max · tools | 88.3%max | Not reported | Not reported |
| DeepSeek V4.1 Flash | 90.9%max | 36.8%max · no tools | 63.9%max · tools | 90.6%max | 3,471max | Not reported |
| Gemma 4 31B | 84.3% | 19.5%no tools | Not reported | Not reported | 2,150 | 85.2% |
| GPT-OSS 120B | 80.1%high · no tools | 14.9%high · no tools | 19.0%high · tools | Not reported | 2,463high · no tools | Not reported |
| Llama 3.3 70B | 50.5%0-shot · CoT | Not reported | Not reported | Not reported | Not reported | 68.9%5-shot · CoT |
| GLM-5.3 | Not reported | Not reported | 62.5%tools | 88.2% | Not reported | Not reported |
Each publisher runs its own tests with its own settings: reasoning level, tools, number of tries and sometimes its own harness. Two scores a few points apart from different publishers aren’t a like-for-like comparison. We haven’t rerun these benchmarks ourselves.
Speed, measured weekly.
Once a week a script sends each model the same three short requests through our developer API, at the reasoning setting the workspace uses by default, and records how long the first word takes and how fast the rest arrives. The figures are medians, with how many answers they come from.
Not measured yet.
The first weekly measurement hasn’t run, so there are no speed figures here. We won’t show an estimate in their place. Until then, each model’s pace on the models page is a class from how it answers by default, not a measurement.
Every score, with its source.
All the figures we copied from each model card, beyond the columns above, and the conditions the publisher states for each.
Kimi K3
- GPQA Diamondmaximum reasoning93.5%
- Humanity’s Last Exam (HLE-Full)maximum reasoning · no tools43.5%
- Humanity’s Last Exam, with tools (HLE-Full)maximum reasoning · with tools56.0%
- Terminal-Bench 2.1maximum reasoning88.3%
- BrowseCompmaximum reasoning91.2%
- OSWorld-Verifiedmaximum reasoning84.8%
- MMMU-Promaximum reasoning · no tools81.6%
Moonshot AI’s Kimi K3 model cardRead 30 October 2026
GLM-5.3
- Humanity’s Last Exam, with toolswith tools62.5%
- Terminal-Bench 2.188.2%
- DeepSWE v1.166.9%
- CyberGymmaximum reasoning84.5%
- Toolathlon Verified73.0%
Z.ai’s GLM-5.3 model cardRead 30 October 2026
DeepSeek V4.1 Flash
- GPQA Diamondmaximum reasoning90.9%
- Humanity’s Last Exammaximum reasoning · no tools36.8%
- Humanity’s Last Exam, with toolsmaximum reasoning · with tools63.9%
- Terminal-Bench 2.1maximum reasoning90.6%
- Codeforcesmaximum reasoning3,471
- DeepSWE v1.1maximum reasoning74.2%
- CyberGymmaximum reasoning88.1%
DeepSeek’s DeepSeek-V4.1-Flash model cardRead 30 October 2026
Gemma 4 31B
- GPQA Diamond84.3%
- Humanity’s Last Examno tools19.5%
- Codeforces2,150
- MMLU-Pro85.2%
- Humanity’s Last Exam, with searchwith web search26.5%
- AIME 2026no tools89.2%
- LiveCodeBench v680.0%
- MMMU-Pro76.9%
- τ²-benchaverage of 3 runs76.9%
Google’s Gemma 4 31B-it model cardRead 30 October 2026
GPT-OSS 120B
- GPQA Diamondhigh reasoning · no tools80.1%
- Humanity’s Last Examhigh reasoning · no tools14.9%
- Humanity’s Last Exam, with toolshigh reasoning · with tools19.0%
- Codeforceshigh reasoning · no tools2,463
- MMLUhigh reasoning90.0%
- SWE-bench Verifiedhigh reasoning62.4%
- AIME 2025high reasoning · no tools92.5%
- τ-bench Retailhigh reasoning67.8%
OpenAI’s gpt-oss-120b & gpt-oss-20b model card (arXiv 2508.10925), table 3Read 30 October 2026
Llama 3.3 70B
- GPQA Diamond0-shot · chain of thought50.5%
- MMLU-Pro5-shot · chain of thought68.9%
- MMLU0-shot · chain of thought86.0%
- HumanEval0-shot88.4%
- IFEval92.1%
- MATH0-shot · chain of thought77.0%
- MGSM0-shot91.1%
Meta’s Llama 3.3 70B Instruct model cardRead 30 October 2026
Sources
- Moonshot AI’s Kimi K3 model cardRead 30 October 2026
- Z.ai’s GLM-5.3 model cardRead 30 October 2026
- DeepSeek’s DeepSeek-V4.1-Flash model cardRead 30 October 2026
- Google’s Gemma 4 31B-it model cardRead 30 October 2026
- OpenAI’s gpt-oss-120b & gpt-oss-20b model card (arXiv 2508.10925), table 3Read 30 October 2026
- Meta’s Llama 3.3 70B Instruct model cardRead 30 October 2026
Scores are copied by hand from each publisher’s model card when a model joins Undergrove, and checked again when the card changes; the weekly job also checks that every page cited here still answers. A score we can’t point to on a publisher’s page isn’t listed.