AA-Briefcase
AA-Briefcase is a private benchmark from Artificial Analysis that tests AI agents on long-horizon office work.
Explore AI Benchmarks through related topics and the articles other pages reference most.
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
Showing 1-39 of 39 articles
AA-Briefcase is a private benchmark from Artificial Analysis that tests AI agents on long-horizon office work.
Agent Arena is a public leaderboard run by Arena (the evaluation company formerly called LMArena
Agent benchmark reward hacking refers to the practice of inflating an AI agent's score on an evaluation suite by attacking the evaluation machinery itself rather than by completing the assigned tasks.
Agent evaluation is the systematic measurement of how well AI agents (LLM-based systems that plan and act over multiple steps using tools) perform on real-world tasks, using benchmarks, metrics, and testing…
AgentBench is a multi-dimensional benchmark for evaluating large language models (LLMs) as autonomous agents across eight distinct interactive environments
AgentDojo is a dynamic evaluation environment for measuring prompt injection attacks and defenses against tool-using large language model agents.
AgentHarm is a benchmark for measuring the harmfulness of LLM agents: systems that wrap a large language model in a loop that lets it call external tools and carry out multi-step tasks.
AutomationBench is a benchmark for AI agents, built by the automation company Zapier, that tests whether a model can finish a multi-step business workflow across several simulated software applications.
BlenderWorld is a benchmark for visual AI agents that build editable 3D scenes in Blender, the open-source 3D creation suite, from text briefs, reference images, reference videos and structured specification…
BountyBench is a cybersecurity benchmark from Stanford University that measures the offensive and defensive capabilities of AI agents on real-world bug-bounty tasks
BrowserGym is an open-source Gymnasium-style environment and unified benchmark ecosystem for web agent research, developed by ServiceNow Research.
Build-A-Chip-Bench is a benchmark for AI agents that tests whether an agent can carry a digital chip design from a workload specification all the way to a routed, verified GDS layout
Crusader Bench is an AI benchmark that tests AI agents by having them develop a settlement's economy in Stronghold Crusader: Definitive Edition.
CursorBench is an agentic coding benchmark built and run by Cursor, the AI code editor made by Anysphere.
DeepSWE is a benchmark for AI coding agents made by Datacurve, a San Francisco data company.
FrontierCode is an agentic coding benchmark created by Cognition, the company behind the Devin coding agent.
GAIA (General AI Assistants) is a benchmark for evaluating general-purpose AI agents and assistants on real-world tasks that require reasoning, web browsing, file handling, and multimodal understanding.
GDPx is an AI benchmark published by SemiAnalysis on November 2, 2026 to evaluate agents working in simulated professional environments.
Harness, in artificial intelligence, is the software scaffolding that wraps a machine learning model, most often a large language model
Intrinsic Discovery is an experimental training method introduced by Induction Labs in August 2026.
MLE-bench is a benchmark for measuring how well autonomous AI agents perform real-world machine learning engineering, released by OpenAI's Preparedness team on October 9, 2024 (arXiv:2410.07095).
MM-BrowseComp is a benchmark for evaluating multimodal web-browsing AI agents, introduced in August 2025 by researchers from ByteDance, Nanjing University, M-A-P, the Institute of Automation of the Chinese…
Mind2Web is a large-scale dataset and benchmark, described by its authors as the first of its kind, for building and evaluating generalist AI agents that follow natural language instructions to complete tasks…
OSWorld is a benchmark for evaluating multimodal AI agents on open-ended tasks performed inside real computer environments.
Online-Mind2Web is a live-web benchmark for measuring whether browser-based AI agents can complete multi-step tasks on real websites.
PaperBench is a benchmark released by OpenAI in April 2025 that measures whether AI agents can replicate cutting-edge machine learning research papers from scratch.
RSI-Exam is a benchmark that tests whether an AI agent can take a working but weak executable method, improve it through hours of autonomous experimentation, and produce a version that still performs better on…
SWE-Atlas is a benchmark for evaluating AI coding agents on professional software-engineering work that goes beyond fixing bugs and resolving issues.
SWE-Bench Pro (stylized SWE-BENCH PRO) is a contamination-resistant benchmark, released by Scale AI in September 2025, that measures whether an AI coding agent can resolve long-horizon
Samaya AI is an American artificial intelligence company that builds AI agents for financial research and analysis, used by Morgan Stanley and a number of hedge funds.
SkillsBench is a benchmark for measuring whether Agent Skills, the structured packages of procedural knowledge that augment AI agents at inference time, actually improve how well those agents do real work.
τ²-bench (also written Tau2-bench or τ^2-bench) is a benchmark for evaluating conversational AI agents in dual-control environments, where both the agent and a simulated user can call tools to read from and…
Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.
Vending-Bench is a family of benchmarks from Andon Labs that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated vending-machine business.
VisualWebArena (often abbreviated VWA) is a benchmark of 910 realistic, visually grounded web tasks for evaluating multimodal autonomous AI agents, released in January 2024 by researchers at Carnegie Mellon…
WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web.
WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024.
Windows Agent Arena (WAA) is a reproducible benchmark and evaluation environment created by Microsoft for testing multimodal computer-use agents on real Windows tasks at scale.
WindowsWorld is a process-aware benchmark for evaluating autonomous computer-use agents on professional workflows that cross several Windows applications.