Multi-SWE-bench
Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.
Model Evaluation
Explore Model Evaluation through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: AI Code Generation
Articles that also belong to these categories. Counts cover all of Model Evaluation.
All articles (134)AI Benchmarks (56)Machine Learning (55)Statistics (15)AI Companies (14)Large Language Models (12)AI Safety (11)Natural Language Processing (9)AI Agents (6)AI Code Generation (6)Developer Tools (6)MLOps (5)AI Ethics (4)Deep Learning (4)Open Source AI (4)Reasoning Models (4)AI Policy & Regulation (3)Computer Vision (3)Data & Datasets (3)AI Infrastructure (2)AI Research (2)AI Tools & Products (2)Information Retrieval (2)Meta AI (2)Reinforcement Learning (2)Training & Optimization (2)2025 in artificial intelligence (1)AI Models (1)Anthropic (1)Artificial Intelligence (1)Data Centers (1)Education AI (1)Finance AI (1)Google DeepMind (1)Interpretability (1)NVIDIA (1)Neural Networks (1)Research Organizations (1)Software Development (1)Speech & Audio AI (1)Video Generation (1)World Models (1)
Showing 1-6 of 6 articles
Pass@k
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
Real-SWE
Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…
SWE-bench Verified
SWE-bench Verified is a 500-problem, human-validated subset of the SWE-bench software engineering benchmark, released on August 13
Spider 2.0
Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows.
Terminal-Bench
Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.