AI Code Generation

Explore AI Code Generation through related topics and the articles other pages reference most.

114 articlesSearch and filter Follow updates by RSS

Explore articles

Browse subtopics: AI Benchmarks

Articles that also belong to these categories. Counts cover all of AI Code Generation.

Showing 1-25 of 25 articles

BigCodeBench

BigCodeBench is a Python code generation benchmark of 1,140 function-level programming tasks that require composing 723 distinct function calls from 139 libraries across seven domains

AI Benchmarks

CRMArena / CRMArena-Pro

CRMArena is an AI benchmark for evaluating large language model agents on professional customer relationship management (CRM) tasks inside a realistic, schema-faithful Salesforce environment.

AI Benchmarks

CRUXEval (Code Reasoning, Understanding, and eXecution Evaluation) is a benchmark designed to measure how well large language models can reason about, understand, and mentally execute short Python programs.

AI Benchmarks Machine Learning

CodeContests

CodeContests is a competitive programming dataset created by Google DeepMind for training and evaluating machine learning models on algorithmic problem-solving tasks.

AI Benchmarks Machine Learning

FeatureBench

FeatureBench is an execution-based benchmark for measuring how well LLM-powered coding agents handle complex, feature-oriented software development rather than bug fixing.

AI Benchmarks

GSO (Global Software Optimization Bench)

GSO (Global Software Optimization), also styled as a benchmark of "Challenging Software Optimization Tasks for Evaluating SWE-Agents," is an AI benchmark that measures whether AI agents and language models can…

AI Benchmarks

HumanEval

HumanEval is a benchmark for measuring whether a code-generating large language model can complete short Python functions so that they pass unit tests. It contains 164 hand-written tasks.

AI Benchmarks

LiveCodeBench

LiveCodeBench is a holistic and contamination-free benchmark for evaluating large language models on code, first released in March 2024 by researchers at UC Berkeley, MIT, and Cornell led by Naman Jain.

AI Benchmarks Machine Learning

MBPP

MBPP (Mostly Basic Python Problems) is a code generation benchmark of 974 crowd-sourced Python programming tasks designed to be solvable by entry-level programmers, introduced by Jacob Austin, Augustus Odena…

AI Benchmarks Large Language Models

Multi-SWE-bench

Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.

AI Benchmarks Model Evaluation

MultiChallenge

MultiChallenge is an AI benchmark for evaluating large language models on realistic multi-turn conversations.

AI Benchmarks

Pass@k

Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.

AI Benchmarks Machine Learning

Real-SWE

Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…

AI Benchmarks Model Evaluation

RepoBench

RepoBench is an AI benchmark for repository-level code auto-completion, introduced in the 2023 paper "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems" by Tianyang Liu, Canwen Xu, and…

AI Benchmarks

SWE-Atlas

SWE-Atlas is a benchmark for evaluating AI coding agents on professional software-engineering work that goes beyond fixing bugs and resolving issues.

AI Agents AI Benchmarks

SWE-Bench Pro

SWE-Bench Pro (stylized SWE-BENCH PRO) is a contamination-resistant benchmark, released by Scale AI in September 2025, that measures whether an AI coding agent can resolve long-horizon

AI Agents AI Benchmarks

SWE-Lancer

SWE-Lancer is a benchmark released by OpenAI in February 2025 that evaluates the ability of frontier large language models to perform real-world freelance software-engineering work.

AI Benchmarks OpenAI

SWE-bench Multilingual

SWE-bench Multilingual is an AI benchmark of 300 real-world software bug-fixing tasks drawn from 42 open-source repositories across nine programming languages

AI Benchmarks

SWE-bench Multimodal

SWE-bench Multimodal (also written SWE-bench M) is a benchmark that measures whether autonomous software-engineering systems can resolve bugs in visual

AI Benchmarks Multimodal AI

SWE-bench Verified

SWE-bench Verified is a 500-problem, human-validated subset of the SWE-bench software engineering benchmark, released on August 13

AI Benchmarks Model Evaluation

SWE-rebench

SWE-rebench is a continuously refreshed, contamination-resistant AI benchmark and public leaderboard for evaluating AI agents on real-world software engineering tasks.

AI Benchmarks

Spider 2.0

Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows.

AI Benchmarks Model Evaluation

Terminal-Bench

Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.

AI Agents AI Benchmarks

TheAgentCompany

TheAgentCompany is an AI benchmark that evaluates AI agents on long-horizon, economically valuable knowledge work inside a self-hosted simulation of a small software company.

AI Benchmarks

Vibe Code Bench

Vibe Code Bench (VCB) is a benchmark that measures whether AI models can build a complete, deployable web application from nothing but a written specification.

AI Benchmarks