Open LLM Leaderboard
The Open LLM Leaderboard was a public ranking of open large language models that Hugging Face ran from 2023 to 2025. Anyone could submit a model hosted on the Hugging Face Hub, and the leaderboard team evaluated it on Hugging Face's own GPU cluster with EleutherAI's LM Evaluation Harness, using the same questions, the same few-shot settings and the same hardware for every entry so that scores could be compared and reproduced.[1][2][13] The first version averaged results on a fixed set of academic benchmarks such as MMLU, HellaSwag and TruthfulQA; a second version, launched on June 26, 2024, replaced them with six harder tests and a normalized score.[1][2] On March 13, 2025, after more than 13,000 models had been evaluated over two years, the maintainers announced that the leaderboard was retiring because it was becoming obsolete as model capabilities changed.[4]
In February 2025 Hugging Face described it as the most used leaderboard on the Hub.[8] Over its life the maintainers also documented its weaknesses in public: scores that depended on implementation details, a benchmark that had to be withdrawn, contaminated models, leaderboard-specific overfitting through model merging of fine-tuned models, and a math parser that undercounted correct answers from whole model families.[2][5][6][8][15]
Origins and purpose
According to Hugging Face, the leaderboard grew out of the company's RLHF team, which had tried to reproduce and compare results from several published models and found it nearly impossible: scores in papers and marketing releases came without reproducible code and were often produced with prompts or evaluation setups tuned to favor the model being reported.[2] The team's answer was to evaluate reference models in exactly the same setup, with the same questions asked in the same order, and publish the results.[2] The leaderboard's own description said that the weekly flood of new models and chatbots, often released with large performance claims, made it hard to tell genuine progress in the open-source community from noise.[3]
The Hugging Face Space that hosted the leaderboard was created on April 17, 2023.[26] Early posts link to it under the HuggingFaceH4 organization, and it later moved to a dedicated open-llm-leaderboard organization.[5][4] In a June 2023 explainer, the maintainers described the leaderboard as a wrapper that runs the EleutherAI harness on the spare cycles of Hugging Face's compute cluster and stores the results in a dataset on the Hub, from which the public Space displays them.[5] Every model's results, per-sample details and submission requests were published as Hub datasets.[3]
The leaderboard ranked only open models. Its FAQ gave two reasons: a philosophical preference for openness, and the practical point that commercial API models can change behind the same name and cannot be re-run by Hugging Face on its own hardware, so their scores could not be kept accurate and reproducible.[13] The suggested citation for the first version credits Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall and Thomas Wolf; the second version credits Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer and Thomas Wolf.[13][27]
By June 2024, Hugging Face said the leaderboard had been visited by more than 2 million unique people over the previous 10 months, and that around 300,000 community members used it each month through submissions and discussions, mostly to find strong open releases or to evaluate their own pretraining and fine-tuning work against existing models.[2]
Timeline
| Date | Event |
|---|---|
| 2023-04-17 | Leaderboard Space created on Hugging Face[26] |
| 2023-06-05 | Hugging Face reports Falcon-40B base and instruct models ranked 1st and 2nd[7] |
| 2023-06-23 | Blog post explains why the leaderboard's MMLU scores differed from published LLaMA numbers[5] |
| 2023-11 | Winogrande, GSM8K and DROP added to the four original benchmarks[6] |
| 2023-12-01 | DROP removed after scoring problems were found[6] |
| 2023-12 | Models trained on data containing TruthfulQA prompts flagged; flagged models hidden by default[14] |
| 2024-02 | Smaug-72B-v0.1 reaches first place with an average above 80[19] |
| 2024-03 | Maintainers explain why merged models are hidden by default[15] |
| 2024-06-26 | Open LLM Leaderboard v2 launched; v1 archived[1][2] |
| 2025-01-09 | Analysis of CO2 emissions from v2 evaluations published[9] |
| 2025-02-14 | All v2 results re-scored with the Math-Verify parser[8] |
| 2025-03-13 | Retirement announced[4] |
Version 1 (2023-2024)
Benchmarks
The leaderboard launched with four benchmarks: the AI2 Reasoning Challenge (ARC), HellaSwag, MMLU and TruthfulQA.[5][6] In November 2023 the maintainers added Winogrande, GSM8K and DROP, using the original implementations reproduced in the EleutherAI harness; Hugging Face said the full re-evaluation that accompanied the change took 8 years of GPU time.[6] DROP was withdrawn a few weeks later (see below), which left the six-benchmark suite that v1 kept until it was archived.[1][6]
| Benchmark | What it tests | Shots | Metric |
|---|---|---|---|
| ARC (Challenge) | Grade-school science questions | 25 | Normalized accuracy |
| HellaSwag | Commonsense inference | 10 | Normalized accuracy |
| MMLU | Multitask knowledge across 57 subjects | 5 | Accuracy (average over subjects) |
| TruthfulQA | Tendency to repeat common falsehoods | 0 | mc2 |
| Winogrande | Commonsense reasoning (Winograd-style) | 5 | Accuracy |
| GSM8K | Grade-school math word problems | 5 | Accuracy |
Source for the table: the v1 archive documentation.[1] The documentation notes that TruthfulQA is technically a 6-shot task in the harness, because each example is prepended with six question-answer pairs even in the "0-shot" setting.[1] The leaderboard ranked models by the average of these scores.[6] All v1 models were evaluated on a single node of eight NVIDIA H100 GPUs.[1]
Models were grouped by type (pretrained, continuously pretrained, fine-tuned on domain-specific data, chat models trained with instruction tuning, RLHF or DPO, and merges) because, in the maintainers' view, these categories did not compete on equal terms.[3][13]
Notable leaders
| Model | When | Note |
|---|---|---|
| Falcon-40B | June 2023 | Base and instruct versions ranked 1st and 2nd when Hugging Face covered the release[7] |
| Platypus | August 2023 | Authors reported first place with a family of fine-tuned and merged models[18] |
| Smaug-72B-v0.1 (Abacus.AI) | February 2024 | Model card describes it as the first open model to average above 80[19] |
The MMLU discrepancy
Soon after Falcon-40B was added in mid-2023, users noticed that the leaderboard's MMLU score for LLaMA, then the top model, was far below the number in the LLaMA paper.[5] Hugging Face investigated and published a June 23, 2023 post showing that three implementations of the same MMLU dataset (the original Berkeley code, Stanford's HELM, and the EleutherAI harness version the leaderboard used) produced very different scores and even different rankings.[5] The original implementation compares the model's probabilities for the answer letters only, HELM compares generated text to the expected answer, and the harness version at the time scored the probability of the full answer sequence.[5] LLaMA 65B scored 0.488 under the harness, against 0.636 under the original implementation and 0.637 under HELM; Falcon-40B scored 0.527 under the harness.[5]
The authors concluded that a bare "MMLU score" says little unless the exact implementation, down to prompts and tokenization, is known.[5] While the post was being written, the harness community updated its MMLU code to match the original implementation, and Hugging Face began re-running the leaderboard with the new version.[5]
DROP withdrawn
After DROP was added in November 2023, most models scored below 10 out of 100 on its F1 metric, a result that did not track their other scores.[6] Hugging Face's own first look found that answers followed by a line break instead of a space escaped the number normalization step, so correct numeric answers failed to match. The Zeno team, which then joined the investigation, traced the remaining failures to the evaluation's use of a period as the end-of-generation token, which cut off decimal answers and let higher-quality models, which imitated the few-shot format, run on past their answer until the next period, lowering their F1.[6] Because the harness followed the official DROP scoring code closely, the fix required a new version of the benchmark's evaluation, and re-running the affected examples would have taken a large amount of GPU time. On December 1, 2023, Hugging Face removed DROP from the leaderboard until a corrected version was available.[6]
Contamination and flagging
The leaderboard let users flag models whose scores looked unfair, for example because the model had been trained on evaluation data or was an unattributed copy of another model; each flagged model was linked to a public discussion where its authors could respond.[13][12] In December 2023 a user reported that the UltraFeedback-binarized and Nectar preference datasets contained TruthfulQA prompts, and that many high-scoring 7B models had either been trained on them or merged from models that had.[14] Nathan Habib of the leaderboard team wrote that the team had tested the models with a work-in-progress contamination tool and found them contaminated on at least TruthfulQA; the models were flagged, and flagged models began to be hidden from the default view.[14] He also noted that testing every model would take a lot of compute, because the individual logits needed for detection had not been saved during evaluation.[14]
In January 2024 Clémentine Fourrier wrote that contamination detection was not a trivial problem and that the team was still investigating detection methods, citing the need to stay fair to evaluated models without methods that would cost days of compute per model, and unresolved questions over what counts as contamination (for instance, whether rephrased test items count).[17] She later argued that the deeper issue was metadata: models were being adopted on leaderboard scores alone, and there was no reliable way to force uploaders to declare their training sets or parent models.[16]
Merged models
By early 2024 model merging had become a common way to climb the leaderboard. One December 2023 discussion complained that the top ranks at every size were held by merged and DPO-tuned models.[28] In March 2024, after a user objected that merges were hidden by default, Fourrier explained the decision: the lineage of merged models (often merges of merges of models that had since been deleted) was nearly impossible to reconstruct, which made them easy to contaminate by accident, and after several rounds of merging and fine-tuning their benchmark results often did not carry over to actual use.[15] The leaderboard FAQ stated that merges had "artificially inflated performance on test sets".[13]
Version 2 (2024-2025)
Why it was rebuilt
Hugging Face launched Open LLM Leaderboard v2 on June 26, 2024, in a post titled "Performances are plateauing, let's make the leaderboard steep again".[2] The maintainers gave three reasons for replacing the v1 benchmarks:[2]
- Saturation. Models were reaching baseline human performance on HellaSwag, MMLU and ARC.
- Contamination. Some newer models showed signs of having been trained on benchmark data or close variants of it, particularly GSM8K and TruthfulQA, which had been included in some instruction-tuning sets.
- Errors. Investigations such as MMLU-Redux and MMLU-Pro had found mistakes in MMLU's answers, and the GSM8K setup used an end-of-generation token that unfairly lowered the scores of verbose models.
The old leaderboard was archived and its results kept available.[1][2]
Benchmarks
The new suite covered knowledge, reasoning over short and long contexts, difficult mathematics, and instruction following, chosen for dataset quality, reliable metrics, low contamination at the time, and relevance to the community.[2] GPQA was chosen partly because its dataset is gated, and MuSR and MMLU-Pro because they were new.[2] The team said it would have liked to include more evaluations, such as MT-Bench, AGIEval and DROP, but was compute-constrained, and it excluded any benchmark that needs another model as a judge.[2]
| Benchmark | What it tests | Shots | Metric |
|---|---|---|---|
| IFEval | Following explicit format instructions | 0 | Strict accuracy (instance and prompt level) |
| BBH (BIG-Bench Hard) | 23 hard BIG-Bench tasks | 3 | Normalized accuracy |
| MATH Level 5 | Hardest competition math problems | 4 | Exact match |
| GPQA | Graduate-level, expert-written science questions | 0 | Normalized accuracy |
| MuSR | Multistep reasoning over long narratives | 0 | Normalized accuracy |
| MMLU-Pro | Harder, 10-option successor to MMLU | 5 | Accuracy |
Source for the table: the leaderboard's About documentation.[3] A later Hugging Face post on the MATH task describes it as 1,324 Level 5 problems across seven topics, and describes a 5-shot setup rather than the 4 shots listed in the documentation.[8][3]
Normalized scoring
Instead of averaging raw scores, v2 rescaled each benchmark so that random guessing scores 0 and a perfect result scores 100, then averaged the rescaled scores.[2] For a multiple-choice task with random-guess baseline $b$ (one over the number of answer options), a raw accuracy $s$ becomes
and 0 when $s < b$. Tasks with subtasks, such as BBH and MuSR, are normalized per subtask and then averaged; generative tasks such as MATH and IFEval have a baseline of 0, so their scores are unchanged.[10] The maintainers pointed out that this changes the effective weight of each benchmark: on raw averages MMLU-Pro looked like one of the two hardest tests, but after normalization GPQA, on which most models were close to chance, joined MATH Level 5 as the hardest.[2]
Other changes
Hugging Face worked with EleutherAI, crediting Hailey Schoelkopf in particular, to update the harness rather than keep v1's pinned older version, whose results had drifted from what users got with newer releases.[2] The update added support for LoRA delta weights, leaderboard-compatible logging and evaluation with chat templates, and the team manually checked task implementations for problems such as inconsistent few-shot samples and overly restrictive end-of-sentence tokens.[2]
Because some top-ranked models were chains of more than 20 successive fine-tuning or merging steps whose recipes were hard to trace, v2 introduced a "maintainer's choice" list of models from established sources.[2] It also replaced the first-submitted, first-evaluated queue with community voting on which models to run first, since the leaderboard ran on spare cycles of the Hugging Face science cluster.[2] A new front end used a Leaderboard component built by the Gradio team.[2] The documentation later described an "Only Official Providers" filter for models from a curated list of trusted providers.[12]
Initial results
At launch, Alibaba's Qwen2-72B-Instruct led with an average of 43.02, followed by Meta's Llama 3-70B-Instruct at 36.67.[2] The maintainers noted that Llama-3-70B-Instruct scored 4.92 on GPQA against 19.67 for its pretrained version, and that some chat models scored well below their base models on MATH, which they attributed to fine-tuning that made models extremely verbose.[2] DeepLearning.AI's newsletter The Batch reported that some models moved as many as 59 places between the old and new rankings.[20]
Emissions estimates and the Math-Verify re-evaluation
The leaderboard added an estimate of the CO2 emitted by each evaluation, computed from the run time, an assumed 5.6 kW draw for eight H100 SXM GPUs, and an average carbon intensity of 269.8 g of CO2 per kWh for electricity in Virginia.[11] A January 2025 analysis of more than 3,000 models evaluated since June 2024 found that larger models generally cost more without proportional score gains, and that community fine-tunes and merges tended to be more CO2-efficient than the official models they started from; the authors cautioned that the figures describe their specific inference setup, not a model's emissions in general.[9]
In February 2025 Hugging Face re-evaluated 3,751 models, all those submitted since the v2 launch in June 2024, with Math-Verify, a new answer parser.[8] The original MATH scoring required answers in a fixed sentence format and then parsed them with SymPy, and it failed on many correct answers, including those written in LaTeX \boxed{} notation, intervals, sets and matrices.[8] On average, models solved 61 more problems after the change, a gain of 4.66 points; Qwen models' MATH scores more than doubled and DeepSeek models' nearly tripled, NVIDIA's AceMath models took over the top of the MATH ranking, and some models rose 200 or more places overall.[8]
Retirement
On March 13, 2025, Fourrier announced in the leaderboard's community forum that it was "officially retiring".[4] The post said the team had evaluated more than 13,000 models over two years on its research cluster, and explained: "The leaderboard is slowly becoming obsolete; we feel it could encourage people to hill climb irrelevant directions in the field."[4] It pointed to the arrival of reasoning models and LM assistants as the change that benchmarks needed to follow, noted that more than 200 community-run leaderboards were by then hosted on Hugging Face, and said the team's future evaluation work would appear under the OpenEvals organization.[4]
The Space stayed online with an "Archived" label next to its title.[27] The v1 results remain in separate archive datasets and an archive Space.[1][13] As of November 18, 2026, the Space had more than 14,000 likes on the Hugging Face Hub.[26]
Criticism and limitations
Much of the criticism of the leaderboard came from its own maintainers and users, in public posts and discussions.
- Implementation sensitivity. The 2023 MMLU investigation showed that the same dataset could produce different scores and rankings depending on prompt format and scoring method, so leaderboard numbers were not directly comparable with figures in papers.[5]
- Benchmark and scoring bugs. DROP had to be withdrawn, the GSM8K setup penalized verbose models (some models, mostly from 01.AI, scored 0 on GSM8K under v1's format), and the v2 MATH parser undercounted correct answers until the Math-Verify fix.[2][6][8]
- Contamination. Test questions leaked into popular training datasets, and merging spread contaminated weights through families of derived models.[2][14]
- Overfitting to the leaderboard. The v2 announcement noted that choosing which models to merge by their scores on the same benchmarks seemed to raise those scores without improving real-world quality.[2] The retirement post said the leaderboard itself risked encouraging work in irrelevant directions.[4]
- Saturation. Top models reached human-baseline scores on several v1 benchmarks within about a year, one of the reasons Hugging Face gave for the v2 rebuild.[2][20]
- Format bias. The maintainers noted that IFEval's format favors chat and instruction-tuned models over pretrained ones.[2]
- Narrow scope. In 2023 Hugging Face wrote that the leaderboard did not measure chat capabilities, for which human evaluation is the gold standard.[7] Closed models were excluded by design.[13]
- Statistical resolution. A September 2026 preprint by Chen Yang and Jun Chen audited 394 claims that one model outranked the next on the Open LLM Leaderboard and reported that 391 lacked statistical support even before accounting for developers selecting among hidden model variants.[23]
Use in research and related leaderboards
Researchers have reused the leaderboard's published results and benchmark suite in work on cheaper evaluation. The tinyBenchmarks project (2024) released small versions of popular benchmarks, including the Open LLM Leaderboard suite, and reported that a model's MMLU performance could be estimated accurately from 100 curated examples.[21] The metabench study (2024) used results from more than 5,000 LLMs on v1's six benchmarks to build a sparse benchmark with less than 3% of the original items.[22]
The format was also copied. The Open Ko-LLM Leaderboard for Korean models describes itself as mirroring the English Open LLM Leaderboard while adding private test sets.[24] A separate 2024 project from VILA-Lab, also called "Open-LLM-Leaderboard", is unrelated to Hugging Face's leaderboard; it evaluates models, including closed ones such as GPT-4o, with open-style rather than multiple-choice questions.[25]
Continue reading
- Data contaminationData contamination is the presence of benchmark evaluation data in the corpus used to train a modelMentioned 7 times above
- HELM (Holistic Evaluation of Language Models)HELM (Holistic Evaluation of Language Models) is an open-source benchmark framework created by the Center for Research on Foundation Models (CRFM) at Stanford University for the reproducible and transparent…Mentioned 3 times above
- LM Evaluation HarnessLM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework for measuring the performance of large language…Listed in See also
- LLM RankingsLLM rankings order large language models from best to worst using one of two broad methods: human preference voting and fixed benchmark scoring.Listed in See also
- LLM EvaluationLLM evaluation is the practice of measuring what a large language model can do, how reliably it does it, and how it behaves under adversarial or high-stakes conditions.Listed in See also
- Chatbot ArenaChatbot Arena (now branded simply as Arena, and previously known as LMArena) is a crowdsourced evaluation platform for large language models that ranks AI systems based on human preferences through anonymous…Listed in See also
More from See also
More to read
Topics:AI BenchmarksLarge Language ModelsModel EvaluationOpen Source AI
References
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Hugging Face. "Open LLM Leaderboard v1". Hugging Face Leaderboards documentation. huggingface.co/...archive
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, Thomas Wolf. "Performances are plateauing, let's make the leaderboard steep again". Hugging Face, June 26, 2024. huggingface.co/...blog
- ^1 ^2 ^3 ^4 ^5Hugging Face. "About" (Open LLM Leaderboard). Hugging Face Leaderboards documentation. huggingface.co/...about
- ^1 ^2 ^3 ^4 ^5 ^6 ^7Clémentine Fourrier. "It's been a wild ride, folks :) (end of the Open LLM Leaderboard)". Open LLM Leaderboard community discussion #1135, Hugging Face, March 13, 2025. huggingface.co/...1135
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Clémentine Fourrier, Nathan Habib, Julien Launay, Thomas Wolf. "What's going on with the Open LLM Leaderboard?". Hugging Face Blog, June 23, 2023. huggingface.co/...open-llm-leaderboard-mmlu
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Clémentine Fourrier, Alex Cabrera, Stella Biderman, Nathan Habib, Thomas Wolf. "Open LLM Leaderboard: DROP deep dive". Hugging Face Blog, December 1, 2023. huggingface.co/...open-llm-leaderboard-drop
- ^1 ^2 ^3Leandro von Werra et al. "The Falcon has landed in the Hugging Face ecosystem". Hugging Face Blog, June 5, 2023. huggingface.co/...falcon
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Hynek Kydlicek, Alina Lozovskaya, Nathan Habib, Clémentine Fourrier. "Fixing Open LLM Leaderboard with Math-Verify". Hugging Face Blog, February 14, 2025. huggingface.co/...math_verify_leaderboard
- ^1 ^2Alina Lozovskaya, Nathan Habib, Albert Villanova del Moral, Clémentine Fourrier. "CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard". Hugging Face Blog, January 9, 2025. huggingface.co/...leaderboard-emissions-analysis
- ^Hugging Face. "Scores Normalization" (Open LLM Leaderboard). Hugging Face Leaderboards documentation. huggingface.co/...normalization
- ^Hugging Face. "CO2 calculation" (Open LLM Leaderboard). Hugging Face Leaderboards documentation. huggingface.co/...emissions
- ^1 ^2Hugging Face. "FAQ" (Open LLM Leaderboard). Hugging Face Leaderboards documentation. huggingface.co/...faq
- ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Hugging Face. "about.py" (About and FAQ text of the Open LLM Leaderboard v1). open-llm-leaderboard-old/open_llm_leaderboard Space, Hugging Face. huggingface.co/...about.py
- ^1 ^2 ^3 ^4 ^5"Potential data contamination with regards to ultrafeedback-binarized and Nectar datasets". Open LLM Leaderboard community discussion #474, Hugging Face, December 16, 2023. huggingface.co/...474
- ^1 ^2 ^3"Disconcerting Perception Surrounding Invisible 'Merge' Due to Open LLM Leaderboard Defaults". Open LLM Leaderboard community discussion #629, Hugging Face, March 11, 2024. huggingface.co/...629
- ^"The Unintended Consequences of Model Merging in the AI Landscape: A Case Study of the Marcoroni-7B-v3 Model". Open LLM Leaderboard community discussion #544, Hugging Face, January 17, 2024. huggingface.co/...544
- ^Clémentine Fourrier. "Discussion thread: Model contamination techniques". Open LLM Leaderboard community discussion #472, Hugging Face, December 15, 2023. huggingface.co/...472
- ^Ariel N. Lee, Cole J. Hunter, Nataniel Ruiz. "Platypus: Quick, Cheap, and Powerful Refinement of LLMs". arXiv:2308.07317, August 14, 2023. arxiv.org/...2308.07317
- ^1 ^2Abacus.AI. "Smaug-72B-v0.1" (model card). Hugging Face, February 2024. huggingface.co/...Smaug-72B-v0.1
- ^1 ^2DeepLearning.AI. "Challenging Human-Level Models: Hugging Face overhauls open LLM leaderboard with tougher benchmarks". The Batch, July 3, 2024. deeplearning.ai/...erboard-with-tougher-benchmarks
- ^Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, Mikhail Yurochkin. "tinyBenchmarks: evaluating LLMs with fewer examples". arXiv:2402.14992, February 22, 2024. arxiv.org/...2402.14992
- ^Alex Kipnis, Konstantinos Voudouris, Luca M. Schulze Buschoff, Eric Schulz. "metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models". arXiv:2407.12844, July 4, 2024. arxiv.org/...2407.12844
- ^Chen Yang, Jun Chen. "How Sensitive Are LLM Leaderboard Claims to Hidden Model Selection?". arXiv:2609.28177, September 23, 2026. arxiv.org/...2609.28177
- ^Chanjun Park, Hyeonwoo Kim, Dahyun Kim, Seonghwan Cho, Sanghoon Kim, Sukyung Lee, Yungi Kim, Hwalsuk Lee. "Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark". arXiv:2405.20574, May 31, 2024. arxiv.org/...2405.20574
- ^Aidar Myrzakhan, Sondos Mahmoud Bsharat, Zhiqiang Shen. "Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena". arXiv:2406.07545, June 11, 2024. arxiv.org/...2406.07545
- ^1 ^2 ^3Hugging Face. Space metadata for open-llm-leaderboard/open_llm_leaderboard (createdAt, likes). Hugging Face Hub API, accessed November 18, 2026. huggingface.co/...open_llm_leaderboard
- ^1 ^2Hugging Face. "Open LLM Leaderboard" Space (front-end source: LeaderboardPage.js and QuotePage.js). Hugging Face, accessed November 18, 2026. huggingface.co/...pages
- ^"Brainstorming: Suggestions for improving the leaderboard". Open LLM Leaderboard community discussion #477, Hugging Face, December 17, 2023. huggingface.co/...477
Improve this article
Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.
1 revision · v1 · 3,881 words · full history
Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify
Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here
Reviewer note: Independent fact-check before publication on November 18, 2026: 112 claims checked against 31 cited and independent sources; 0 material and 7 minor corrections applied before publication.
Cite this page: AI Wiki. "Open LLM Leaderboard." aiwiki.ai, updated 18 Nov 2026, fact-checked 18 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/open_llm_leaderboard
