AI Benchmarks

Explore AI Benchmarks through related topics and the articles other pages reference most.

Explore articles

Browse subtopics (56)

Articles that also belong to these categories. Counts cover all of AI Benchmarks.

Showing 241-268 of 268 articles

ToxiGen

ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection.

AI EthicsAI Safety

TriviaQA

TriviaQA is a large-scale reading comprehension and question answering dataset of over 650,000 question-answer-evidence triples, introduced in 2017 by Mandar Joshi, Eunsol Choi, Daniel S. Weld

Natural Language Processing

Vending-Bench

Vending-Bench is a family of benchmarks from Andon Labs that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated vending-machine business.

AI AgentsAI Safety

Vibe Code Bench

Vibe Code Bench (VCB) is a benchmark that measures whether AI models can build a complete, deployable web application from nothing but a written specification.

AI Code Generation

Video-MME

Video-MME (Video Multi-Modal Evaluation) is a benchmark for testing how well multimodal large language models (MLLMs) understand video, built from 900 manually selected videos totaling 254 hours and 2,700…

Computer VisionMultimodal AI

Video-MMMU

Video-MMMU (Video Multi-Modal Multi-disciplinary Understanding, sometimes written VideoMMMU) is a benchmark that measures whether Large Multimodal Models can acquire new knowledge from professional educational…

Education AI

Vimgolf

vimgolf-gym is an OpenAI Gym style customizable environment and benchmark built around VimGolf, the long running keystroke counting puzzle game for Vim.

VisualWebArena

VisualWebArena (often abbreviated VWA) is a benchmark of 910 realistic, visually grounded web tasks for evaluating multimodal autonomous AI agents, released in January 2024 by researchers at Carnegie Mellon…

AI Agents

WISE

WISE (World Knowledge-Informed Semantic Evaluation) is an AI benchmark that tests whether a text-to-image model actually possesses and correctly applies real-world knowledge when it draws a scene

Computer Vision

WMDP benchmark

The Weapons of Mass Destruction Proxy (WMDP) is a publicly released benchmark and unlearning testbed for large language models, introduced in the paper "The WMDP Benchmark: Measuring and Reducing Malicious Use…

AI Safety

WebArena

WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web.

AI AgentsModel Evaluation

WebDev Arena

WebDev Arena is a live, community-driven leaderboard that ranks large language models on their ability to generate working web applications.

WebVoyager

WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024.

AI AgentsModel Evaluation

WeirdML

WeirdML is a benchmark for evaluating how well large language models can do hands on machine learning engineering.

Developer Tools

WildBench

WildBench is an automated evaluation framework for large language models (LLMs) introduced by the Allen Institute for AI (AI2) in 2024.

Model Evaluation

Windows Agent Arena

Windows Agent Arena (WAA) is a reproducible benchmark and evaluation environment created by Microsoft for testing multimodal computer-use agents on real Windows tasks at scale.

AI AgentsMicrosoft

WindowsWorld

WindowsWorld is a process-aware benchmark for evaluating autonomous computer-use agents on professional workflows that cross several Windows applications.

AI Agents

WinoGrande

WinoGrande is a large-scale benchmark for commonsense reasoning consisting of 44,000 binary fill-in-the-blank pronoun resolution problems, built to test whether language models genuinely understand commonsense…

Natural Language Processing

ZeroBench

ZeroBench is a visual reasoning benchmark built to be effectively impossible for current frontier large multimodal models, which score 0.0% on its main questions.

Computer VisionMultimodal AI

τ-bench

τ-bench (Tau-bench), short for Tool-Agent-User Interaction Benchmark, is an AI benchmark that evaluates language agents' ability to complete complex tasks through realistic, multi-turn interactions with…