Showing 181-220 of 220 articles
ScienceAgentBench
ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing...
Model Evaluation
SimpleBench
SimpleBench is a text-only benchmark for large language models created by Philip, the host of the AI Explained YouTube channel, with collaborator Hemang. It...
Artificial IntelligenceNatural Language Processing
SimpleQA
SimpleQA is a factuality benchmark released by OpenAI on October 30, 2024 that measures whether large language models can answer short, fact-seeking questions...
AI SafetyNatural Language Processing
SimpleQA Verified
SimpleQA Verified is a short-form factuality benchmark released by Google DeepMind and Google Research in September 2025 that measures the parametric knowledge...
Google DeepMindModel Evaluation
SkillsBench
SkillsBench is a benchmark for measuring whether Agent Skills, the structured packages of procedural knowledge that augment AI agents at inference time,...
AI Agents
Spider 2.0
Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows. Introduced by Fangyu Lei and colleagues in the...
AI Code GenerationModel Evaluation
StrongREJECT
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models. It was introduced...
AI SafetyModel Evaluation
SuperGLUE
SuperGLUE is a public benchmark for evaluating general-purpose English language understanding in machine learning systems, consisting of eight challenging...
Data & DatasetsNatural Language Processing
SuperGPQA
SuperGPQA is a large graduate-level knowledge and reasoning benchmark for evaluating large language models across 285 academic disciplines. It contains 26,529...
Model Evaluation
T2I-CompBench
T2I-CompBench is an AI benchmark for evaluating compositional text-to-image generation, introduced in the paper "T2I-CompBench: A Comprehensive Benchmark for...
Computer Vision
Task-completion time horizon (METR)
The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of...
AI SafetyModel Evaluation
Tau2-bench
τ²-bench: Evaluating Conversational Agents in a Dual-Control Environment Abbreviation 2025-06-12 (v0.1.0) Latest version arXiv:2506.07982 (2025-06-09) ...
AI Agents
Terminal-Bench
Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces. Developed as a...
AI AgentsAI Code Generation
TheAgentCompany
TheAgentCompany is an AI benchmark that evaluates AI agents on long-horizon, economically valuable knowledge work inside a self-hosted simulation of a small...
AI Code Generation
Tokens per second
Tokens per second (TPS) is a key performance metric for measuring the speed of large language model (LLM) inference. It quantifies how many tokens a model can...
AI HardwareLarge Language Models
ToxiGen
ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection. Created by Thomas Hartvigsen, Saadia Gabriel,...
AI EthicsAI Safety
TriviaQA
TriviaQA is a large-scale reading comprehension and question answering dataset of over 650,000 question-answer-evidence triples, introduced in 2017 by Mandar...
Natural Language Processing
TruthfulQA
TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions. Created by Stephanie Lin, Jacob...
AI SafetyMachine Learning
Vending-Bench
Vending-Bench is a benchmark that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated...
AI Agents
Vibe Code Bench
Vibe Code Bench (VCB) is a benchmark that measures whether AI models can build a complete, deployable web application from nothing but a written specification....
AI Code Generation
Video-MME
Video-MME (Video Multi-Modal Evaluation) is a benchmark for testing how well multimodal large language models (MLLMs) understand video, built from 900 manually...
Computer VisionMultimodal AI
Video-MMMU
Video Multi-Modal Multi-disciplinary Understanding Abbreviation A multi-modal benchmark that evaluates how Large Multimodal Models acquire and apply...
Education AI
Vimgolf
Vimgolf Gym Environment Abbreviation OpenAI Gym style customizable environment and benchmark for VimGolf challenges, used to evaluate AI agents and large...
Visual Question Answering Models
Visual question answering models are AI systems that take an image and a natural language question about that image and return a natural language answer. They...
AI ModelsComputer Vision
VisualWebArena
VisualWebArena (often abbreviated VWA) is a benchmark of 910 realistic, visually grounded web tasks for evaluating multimodal autonomous AI agents, released in...
AI Agents
WISE
WISE (World Knowledge-Informed Semantic Evaluation) is an AI benchmark that tests whether a text-to-image model actually possesses and correctly applies...
Computer Vision
WMDP benchmark
The Weapons of Mass Destruction Proxy (WMDP) is a publicly released benchmark and unlearning testbed for large language models, introduced in the paper "The...
AI Safety
WebArena
WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web....
AI AgentsModel Evaluation
WebDev Arena
Web Development Arena Abbreviation A live, community-driven leaderboard that ranks large language models on their ability to build interactive web...
WebVoyager
WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024 [1][2]. It is...
AI AgentsModel Evaluation
WeirdML
WeirdML is a benchmark for evaluating how well large language models can do hands on machine learning engineering. It was created by Norwegian researcher...
Developer Tools
WildBench
WildBench is an automated evaluation framework for large language models (LLMs) introduced by the Allen Institute for AI (AI2) in 2024. It evaluates models on...
Model Evaluation
Windows Agent Arena
Windows Agent Arena (WAA) is a reproducible benchmark and evaluation environment created by Microsoft for testing multimodal computer-use agents on real...
AI AgentsMicrosoft
WindowsWorld
WindowsWorld is a process-centric benchmark for evaluating autonomous GUI agents on professional, cross-application desktop workflows. Introduced in an April...
AI Agents
WinoGrande
WinoGrande is a large-scale benchmark for commonsense reasoning consisting of 44,000 binary fill-in-the-blank pronoun resolution problems, built to test...
Natural Language Processing
Winograd Schema Challenge
The Winograd Schema Challenge (WSC) is a commonsense reasoning test in which a system must resolve an ambiguous pronoun in a short sentence where the correct...
Natural Language Processing
WritingBench
WritingBench is a comprehensive benchmark for evaluating the generative writing capabilities of large language models (LLMs) across diverse real-world writing...
Large Language ModelsNatural Language Processing
ZebraLogic
ZebraLogic is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs). Developed by researchers at the Allen Institute...
Large Language ModelsNatural Language Processing
ZeroBench
ZeroBench is a visual reasoning benchmark built to be effectively impossible for current frontier large multimodal models, which score 0.0% on its main...
Computer VisionMultimodal AI
τ-bench
τ-bench (Tau-bench), short for Tool-Agent-User Interaction Benchmark, is an AI benchmark that evaluates language agents' ability to complete complex tasks...