MultiLangBench
MultiLangBench is an AI coding benchmark introduced by OpenAI on October 19, 2026. It evaluates implementations of a SQLite-compatible database engine across 20 programming languages.
Explore AI Benchmarks through related topics and the articles other pages reference most.
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
Showing 181-240 of 268 articles
MultiLangBench is an AI coding benchmark introduced by OpenAI on October 19, 2026. It evaluates implementations of a SQLite-compatible database engine across 20 programming languages.
NVIDIA Isaac Lab-Arena is an open-source framework for composing simulated robot tasks and evaluating learned policies at scale.
The NVIDIA Nemotron Model Reasoning Challenge was a Kaggle competition run by NVIDIA from March to June 2026 in which participants tried to improve the reasoning accuracy of a fixed open model, Nemotron 3 Nano…
Natural language inference (NLI), also known as recognising textual entailment (RTE), is the natural language processing task of deciding whether a hypothesis sentence is entailed by, contradicts, or is…
Needle in a Haystack (NIAH) is a long-context evaluation that measures whether a large language model can retrieve a single fact (the "needle") inserted at a controlled position inside a long body of text (the…
NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…
OSWorld is a benchmark for evaluating multimodal AI agents on open-ended tasks performed inside real computer environments.
OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.
Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…
Online-Mind2Web is a live-web benchmark for measuring whether browser-based AI agents can complete multi-step tasks on real websites.
OpenAI MRCR (Multi-Round Co-reference Resolution) is a long-context evaluation dataset published by OpenAI that measures a language model's ability to distinguish between multiple near-identical "needles"…
PASCAL VOC (Pattern Analysis, Statistical Modelling and Computational Learning Visual Object Classes) is a long-running benchmark dataset and annual challenge for object recognition, object detection…
PIQA (Physical Interaction Question Answering) is a benchmark dataset of roughly 21,000 binary multiple-choice questions that evaluates the physical commonsense reasoning abilities of natural language…
Paper2Video (full title: Paper2Video: Automatic Video Generation from Scientific Papers) is a research project from Show Lab at the National University of Singapore that formalizes and evaluates automatic…
PaperBench is a benchmark released by OpenAI in April 2025 that measures whether AI agents can replicate cutting-edge machine learning research papers from scratch.
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
PhyGenBench (Physics Generation Benchmark) is a benchmark for testing whether text-to-video models produce videos that follow basic physical commonsense.
Physics-IQ is a benchmark that tests whether generative video models understand physical principles, by asking them to continue real videos of physical experiments and comparing the result with what actually…
Physics-IQ Verified is an audited and re-scored version of Physics-IQ, the Google DeepMind benchmark that tests whether video generation models can predict how real physical experiments unfold.
ProcessBench is a benchmark for step-level verification of mathematical reasoning, built by the Qwen Team at Alibaba and released in December 2024.
PubMedQA is a biomedical question answering dataset and benchmark that evaluates whether machine learning models can answer yes/no/maybe research questions using evidence from PubMed abstracts.
PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.
RE-Bench (short for Research Engineering Benchmark) is a benchmark for evaluating the frontier AI research-and-development capabilities of large language model agents, developed by the nonprofit METR (Model…
RSI-Exam is a benchmark that tests whether an AI agent can take a working but weak executable method, improve it through hours of autonomous experimentation, and produce a version that still performs better on…
RULER is a synthetic benchmark from NVIDIA that measures the real, usable context window of large language models (LLMs) by testing them on 13 tasks across four categories (retrieval, multi-hop tracing…
Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…
RepoBench is an AI benchmark for repository-level code auto-completion, introduced in the 2023 paper "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems" by Tianyang Liu, Canwen Xu, and…
RewardBench is a benchmark and public leaderboard for evaluating reward models, the scoring functions that sit at the center of reinforcement learning from human feedback (RLHF).
RoboArena is a distributed, real-world benchmark for generalist robot policies in which volunteer evaluators at different laboratories run double-blind, pairwise (A/B) comparisons of policies on tasks and…
RoboChallenge is an online benchmarking platform that evaluates robot control policies, mainly vision-language-action models (VLAs), on physical robots instead of in simulation.
RoboDojo is a benchmark for generalist robot manipulation policies that combines a simulation suite of 42 tasks with a real-robot suite of 18 tasks, a standardized physical test platform called…
Robocurve (Robocurve, Inc., which styles itself Robocurve PBC) is a San Francisco public benefit corporation that evaluates how well frontier AI models and robot-control models perform on real robots, and…
SIMPLER (Simulated Manipulation Policy Evaluation for Real Robot Setups) is a collection of simulated robot manipulation environments, released in 2024
SQuAD (the Stanford Question Answering Dataset) is a large-scale reading comprehension benchmark from Stanford University in which a model must answer a question by extracting the exact span of text that…
SUPERB, which stands for Speech processing Universal PERformance Benchmark, is a comprehensive evaluation framework designed to measure how well self-supervised learning (SSL) models generalize across a…
SWE-Atlas is a benchmark for evaluating AI coding agents on professional software-engineering work that goes beyond fixing bugs and resolving issues.
SWE-Bench Pro (stylized SWE-BENCH PRO) is a contamination-resistant benchmark, released by Scale AI in September 2025, that measures whether an AI coding agent can resolve long-horizon
SWE-Lancer is a benchmark released by OpenAI in February 2025 that evaluates the ability of frontier large language models to perform real-world freelance software-engineering work.
SWE-bench is an execution-based benchmark for evaluating whether a language-model system can resolve real software issues.
SWE-bench Multilingual is an AI benchmark of 300 real-world software bug-fixing tasks drawn from 42 open-source repositories across nine programming languages
SWE-bench Multimodal (also written SWE-bench M) is a benchmark that measures whether autonomous software-engineering systems can resolve bugs in visual
SWE-bench Verified is a 500-problem, human-validated subset of the SWE-bench software engineering benchmark, released on August 13
SWE-rebench is a continuously refreshed, contamination-resistant AI benchmark and public leaderboard for evaluating AI agents on real-world software engineering tasks.
Samaya AI is an American artificial intelligence company that builds AI agents for financial research and analysis, used by Morgan Stanley and a number of hedge funds.
SciCode is a research coding benchmark that asks large language models to write Python code for realistic, PhD level scientific problems drawn from working scientists' day to day workflows.
ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing code.
SimpleBench is a text-only benchmark for large language models created by Philip, the host of the AI Explained YouTube channel, with collaborator Hemang.
SimpleQA is a factuality benchmark released by OpenAI on October 30, 2024 that measures whether large language models can answer short, fact-seeking questions correctly instead of producing hallucinations.
SimpleQA Verified is a short-form factuality benchmark released by Google DeepMind and Google Research in September 2025 that measures the parametric knowledge of large language models using roughly 1,000…
SkillsBench is a benchmark for measuring whether Agent Skills, the structured packages of procedural knowledge that augment AI agents at inference time, actually improve how well those agents do real work.
Specific Labs is a San Francisco company that buys and licenses operational data and source code from businesses, and packages it as training and evaluation material for AI labs.
Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows.
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.
SuperGLUE is a public benchmark for evaluating general-purpose English language understanding in machine learning systems, consisting of eight challenging reasoning and reading-comprehension tasks, two…
SuperGPQA is a large graduate-level knowledge and reasoning benchmark for evaluating large language models across 285 academic disciplines.
T2I-CompBench is an AI benchmark for evaluating compositional text-to-image generation, introduced in the paper "T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation"…
The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of task
τ²-bench (also written Tau2-bench or τ^2-bench) is a benchmark for evaluating conversational AI agents in dual-control environments, where both the agent and a simulated user can call tools to read from and…
Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.
TheAgentCompany is an AI benchmark that evaluates AI agents on long-horizon, economically valuable knowledge work inside a self-hosted simulation of a small software company.