Artificial Analysis

Artificial Analysis is an independent benchmarking and analytics platform that evaluates artificial intelligence models and API providers across intelligence, speed, price, and latency, and it is best known for the Artificial Analysis Intelligence Index, a composite 0-100 score of model quality.^[1][3] Founded in 2023 by Micah Hill-Smith and George Cameron, the platform runs its own evaluations rather than relying on numbers self-reported by AI labs, and it has become one of the most widely referenced independent sources for comparing large language models (LLMs), image, video, text-to-speech, speech-to-text, and embedding models.^[1][16] As of September 2026 the platform carried Intelligence Index data for 644 language models and covered more than 500 API endpoints, served both individual developers and enterprises through free public leaderboards alongside paid subscriptions, and was backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng through AI Grant.^{[5][16][17][22]} "In 2023, there was nothing that we could find where anyone was doing good independent benchmarking of LLMs," co-founder Micah Hill-Smith has said of the gap the platform set out to fill.^[16]

History and Founding

Who founded Artificial Analysis?

Micah Hill-Smith and George Cameron first met while interning at Google. Both went on to pursue separate careers: Hill-Smith worked as a Business Analyst at McKinsey & Company before leaving to build a legal AI startup, while Cameron worked as a Senior Strategy Consultant at Altman Solon, focusing on technology and data centers.^[16]

In 2023, while building applications with AI models, both founders noticed a significant gap in the market. There was no reliable, independent resource for benchmarking LLMs across quality, speed, and price.^[16] AI labs frequently reported their own benchmark results using inconsistent methodologies. A notable example that motivated the founders was Google's reporting of Gemini 1.0 Ultra performance on MMLU, where Google used 32-shot chain-of-thought prompts to make the model appear superior to GPT-4, a methodology that differed from how other labs reported their scores.^[16] Hill-Smith framed the underlying realization plainly: "the more you go into building something using LLMs, the more each bit of what you're doing ends up being a benchmarking problem."^[16]

Hill-Smith and Cameron built Artificial Analysis as a side project from a basement in Sydney, Australia in 2023, initially expecting it to require only occasional updates.^[16] However, the rapid pace of new model releases and strong developer interest quickly outpaced their expectations. The platform launched publicly in January 2024 and gained significant traction after being featured on the Latent Space podcast.^[16] Developers embraced the tool so enthusiastically that both founders decided to pause their other projects and commit to Artificial Analysis full-time.^[16]

How is Artificial Analysis funded?

The pair joined AI Grant's fourth batch, an accelerator program run by Nat Friedman (former CEO of GitHub) and Daniel Gross, and relocated to San Francisco. AI Grant provided a $250,000 investment via an uncapped SAFE plus cloud compute credits and access to a network of prominent AI investors and entrepreneurs, and Artificial Analysis was one of the few AI Grant companies to raise a full seed round from Friedman and Gross.^[16] Andrew Ng also backed the company. As of late 2025, the team had grown to approximately 20 employees.^[17]

Core Mission

Artificial Analysis positions itself as an independent third party in the AI evaluation space. The platform runs its own evaluations rather than relying on self-reported numbers from AI labs.^[1][2] This independence is central to its value proposition: developers and enterprises need trustworthy, standardized data to make informed decisions about which models and providers to use.^[1] Co-founder George Cameron has summarized the policy directly: "you can't pay us for better results."^[16] Hill-Smith has compared independently reproducing benchmarks to consumer reporting rather than an accusation against the labs: "None of this is dodgy or bad. It's like Apple saying how long your MacBook's battery is going to last."^[17]

The platform addresses several practical questions that developers face when building AI applications: Which model offers the best quality for a given task? Which API provider delivers the fastest response times? How much does it cost to run inference at scale? How do different providers compare when handling concurrent requests?^[1]

LLM Leaderboard

The LLM Leaderboard is the flagship product of Artificial Analysis, comparing over 250 AI models across intelligence, price, performance, and speed.^[4] The leaderboard covers both proprietary models from companies like OpenAI, Anthropic, and Google, as well as open-source and open-weight models.^[4]

Which models are tracked?

The leaderboard includes models from all major AI labs and many smaller ones. Proprietary models tracked include the GPT series from OpenAI, Claude models from Anthropic, Gemini models from Google, and others.^[4] Open-weight models include the Llama series from Meta, models from Mistral AI, Qwen models from Alibaba, DeepSeek models, and many more.^[4] As the AI landscape has expanded, the number of frontier labs tracked on the platform has grown from four or five at launch to over thirteen, and by September 2026 the site's Intelligence Index covered 644 language models, of which the main LLM Leaderboard ranks over 250.^[4][17][22]

Model classes and price bands

Rankings on the site are not computed against the whole catalogue. Each model page compares a model only against its own class, and those class rules are what a claim of the form "the highest of any open weights model under 4B total parameters" actually rests on.^[33][38] Non-reasoning models are compared only with other non-reasoning models, while reasoning models are compared across both. Open-weights models are then compared only with other open-weights models of the same size class, and proprietary models are compared across proprietary and open-weights models in the same price band, using a blended 3:1 input/output price ratio.^[33]

Model pages also separate measured results from provisional ones. A score carrying the label "Estimate (independent evaluation forthcoming)" has not yet come out of a completed independent run.^[33]

Key Performance Metrics

Artificial Analysis measures several performance indicators for each model and provider combination:^[2]

For reasoning models that perform internal deliberation before generating a response, the platform also measures Time to First Answer Token, which excludes the thinking phase to provide a more meaningful latency figure.^[2]

Pricing Methodology

Pricing data tracks both input and output token costs separately. To simplify comparisons, the platform publishes a "blended price," which as of 2026 assumes a 7:2:1 ratio of cache-hit, input, and output tokens.^[2][23] Prices are reported per million tokens.^[2]

Because reasoning models consume very different numbers of tokens for the same job, the leaderboards also report Cost per Task, a weighted-average dollar cost of completing one Intelligence Index task, calculated from measured token consumption across the evaluation workload divided by the number of tasks.^[23]

Token Standardization

Different models use different tokenizers, which means the same text can produce different token counts across models. To ensure fair comparisons, Artificial Analysis uses OpenAI tokens as a standard unit of measurement through the tiktoken package (o200k_base tokenizer).^[2] The platform also tracks native token counts separately, since providers typically charge based on their own tokenization.^[2][23]

Measurement Methodology

Performance metrics are collected through real API calls to production endpoints. The platform tests every API endpoint eight times per day for single-request benchmarks and twice per day for parallel-request benchmarks.^[2] Reported figures represent the median measurement over the prior 14 days, with P5, P25, P75, and P95 percentile values also available.^[2]

The leaderboard supports six different workload configurations, varying across two dimensions:^[2]

This allows developers to evaluate provider performance under conditions that match their specific use case, whether that involves short conversational exchanges or long document processing with high concurrency.

Mystery Shopper Policy

To prevent providers from giving preferential treatment to known benchmarking accounts, Artificial Analysis registers evaluation accounts without using their own domain.^[2] This "mystery shopper" approach ensures that the measured performance reflects what an ordinary customer would experience, rather than a specially optimized environment.^[2]

LLM API Providers Leaderboard

In addition to comparing models, Artificial Analysis maintains a separate Providers Leaderboard that compares over 500 API endpoints across different hosting services.^[5] This leaderboard helps developers understand how the same model performs differently depending on where it is hosted.^[5]

Which API providers are benchmarked?

The platform benchmarks a wide range of API providers:^[5]

For each provider, the platform reports throughput, latency, pricing, and availability data, enabling developers to choose between first-party APIs (which may offer the latest model versions first) and third-party providers (which may offer better price-performance ratios).^[5]

Intelligence Benchmarking

The Artificial Analysis Intelligence Index (AAII) provides a composite measure of model quality, synthesizing multiple evaluation benchmarks into a single score on a 0-100 scale.^[3] The index is versioned and updated as benchmarks saturate. It ran as v4.0.2 in January 2026 and advanced to version 4.1 in June 2026, which incorporated nine evaluations and introduced updates to GDPval-AA v2, tau-cubed-Banking, and Terminal-Bench v2.1.^[3][23] The index then moved twice inside a single week in September 2026: v4.2 was announced on 4 September and v4.3 on 7 September, and v4.3 is the current version.^[3][31][32]

Artificial Analysis presented both as interim releases pulled forward from a larger v5 rewrite, saying it was "accelerating elements of our upcoming v5 release with interim updates to keep pace with the frontier" after deliberately holding the index steady through a run of major model launches.^[32] The stated direction of both updates was harder agentic tasks and more held-out data. Evaluations with private questions or answers account for 45% of the v4.3 weighting, up from 40% in v4.2, which was itself double the v4.1 figure.^[31][32]

What is in the Intelligence Index v4.3?

Intelligence Index v4.3 combines ten evaluations in four weighted categories: Agents 30%, Coding 20%, General 30%, and Scientific Reasoning 20%. Those category weights were set in v4.2 and left unchanged in v4.3.^[3][31]

The SciCode figure is the benchmark's test set; the published SciCode dataset contains 338 subproblems decomposed from 80 main problems.^[3][39] The index is a primarily text-based, English-language suite, and Artificial Analysis benchmarks image input, speech input and multilingual ability separately from it.^[3]

Version history of the index

Artificial Analysis publishes a dated changelog for every index version. The v4 line has run as follows.^[3]

The evaluations added in v4.2 and v4.3

Four evaluations entered the index in September 2026. One of them, AA-Briefcase, is Artificial Analysis' own and is covered in its own section below. The other three came from outside the company: AutomationBench from Zapier, GDP.pdf from Surge AI, and Terminal-Bench from Stanford University researchers, the Laude Institute and open-source contributors.^[3][31][32]

AutomationBench-AA is Artificial Analysis' run of Zapier's AutomationBench, which tests whether an agent can complete SaaS workflows spanning several simulated business applications by discovering and calling REST endpoints rather than using purpose-built tools.^[3][35] Artificial Analysis evaluates a private held-out split of 657 tasks from dataset version 1.0.6, covering finance, HR, marketing, operations, sales and support, in simulated environments modelled on products such as Gmail, Google Sheets, Slack, Salesforce, Zendesk, Jira and HubSpot, with a 50-turn cap and one run per task.^[3] Every assertion in a task is classified either as an objective the agent must make true or as a guardrail that starts out satisfied and must not be broken, and both are checked programmatically against the final environment state, with no LLM judge involved. A task that trips any guardrail scores zero; otherwise it scores the share of objectives completed.^[3] Zapier's repository ships a public 600-task set and states that the official leaderboard runs on a separate private set, which "follows a similar task distribution and assertion framework as the public set but is purposely harder and is not released."^[34]

Terminal-Bench v4.0 replaced Terminal-Bench v2.1 in the Coding category at the same 10% weight. Artificial Analysis runs all 66 tasks in the v4.0 dataset three times through the mini-SWE-agent v2.4.6 harness and reports average pass@1, capping the agent at 500 steps and each command at 30 seconds.^[3] Each task carries its own verification suite, grading runs in a separate verifier container isolated from the agent's environment, and a task passes only if every test passes.^[3] The public Terminal-Bench 4.0 leaderboard is hosted by Stanford, Harbor and the Laude Institute.^[37]

GDP.pdf, added in v4.2, is Artificial Analysis' implementation of a Surge AI benchmark on reasoning over long professional documents. The evaluation set is 100 tasks across ten professional domains, grounded in 4,592 PDF pages and graded against 1,275 criteria, attempted five times each for a fixed denominator of 500 attempts per model.^[3][32][36] Artificial Analysis extracts every page as text, using OCR where needed, and additionally supplies page images to models that accept image input, rather than using provider document-input features, which it argues "sit above the model layer" and would mix API product decisions into a model comparison.^[3] The headline metric, All-pass, is the share of the 500 attempts on which every criterion passes; a task-macro Mean Pass rate is reported alongside it.^[3]

Artificial Analysis explained one of these names when it announced v4.3: "We call our implementation AutomationBench-AA because we award partial credit for completed objectives, with any guardrail violation reducing the task's score to zero."^[31] The same suffix appears on GDPval-AA, Harvey LAB-AA, APEX-Agents-AA, ITBench-AA, EnterpriseOps-Gym-AA and MLCR-AA, each of which the methodology page describes as an Artificial Analysis implementation of a benchmark built by someone else.^[3] Plenty of outside benchmarks in the suite carry no suffix, however: Terminal-Bench v4.0, SciCode, HLE, CritPt, GDP.pdf, IFBench, Global-MMLU-Lite and MMMU Pro all keep their original names, and Terminal-Bench v4.0 in particular runs on a harness, a step cap and a command timeout that Artificial Analysis chose itself.^[3]

What left the index

The September updates removed three constituents. GPQA Diamond, part of the index through v4.1.1, was dropped in v4.2, described in the announcement as "an exceptional scientific reasoning evaluation that has now been saturated"; its methodology entry says Artificial Analysis still runs it on new model releases and reports it as a standalone evaluation.^[3][32] GPQA Diamond is the 198-question "diamond subset" on which PhD-level domain experts score about 65% while skilled non-experts with web access reach roughly 34%, making it a deliberately expert-gated science benchmark.^[21]

Terminal-Bench v2.1 was superseded by v4.0 in v4.3. Tau-cubed-Banking, the Sierra AI fintech-support domain of the tau-bench family, was replaced by AutomationBench-AA at the same 5% weight and now appears in the additional-evaluations table, as does IFBench, which had already left the index in v4.1. GPQA Diamond and Terminal-Bench v2.1 are not in that table.^[3][31]

What happens to a retired evaluation after that is left unclear by the methodology page itself. GPQA Diamond, tau-cubed-Banking and Terminal-Bench v2.1 are all filed under a heading called "Legacy Evaluations", whose text says the company keeps them "for reference and historical comparability" and that "they are no longer part of the Artificial Analysis Intelligence Index or our active reporting". The status line on each of the three rows underneath says something narrower: GPQA Diamond and tau-cubed-Banking both carry "We still run it on new model releases and report it as a standalone evaluation", and Terminal-Bench v2.1 carries "It remains part of the Coding Index". The page does not reconcile the heading with the rows.^[3]

Prior versions of the index

The v4.1 framework, current from June to August 2026, used a weighted four-category structure spanning nine evaluations, with a heavier tilt toward agentic tasks than the equal-weight scheme that preceded it:^[3]

The v4 framework introduced in January 2026 had instead weighted Agents, Coding, General, and Scientific Reasoning at 25% each, with evaluations including tau-squared-Bench Telecom (114 agent-user simulation tasks), Terminal-Bench Hard (44 terminal tasks), SciCode (338 scientific computing subproblems), AA-LCR (100 long-context reasoning questions at about 100k tokens each), IFBench (294 instruction-following questions), HLE (2,158 frontier academic questions), GPQA Diamond (198 questions), and CritPt (70 physics reasoning challenges).^[3]

Evaluation Standards

All models undergo identical testing conditions. The methodology emphasizes standardization, avoidance of bias, zero-shot instruction prompting, and full transparency.^[3] Scoring predominantly uses pass@1 metrics, where models must succeed on their first attempt. Multiple runs are aggregated into single pass@1 scores. Artificial Analysis states that it estimates "a 95% confidence interval for Artificial Analysis Intelligence Index of less than plus or minus 1%," based on experiments with more than ten repeats on certain models across all v4.3 evaluations, and notes that individual evaluations may have wider intervals than the composite.^[3]

Common settings apply across the suite: temperature 0 for non-reasoning models and 0.6 for reasoning models unless the lab recommends otherwise, a 16,384-token output cap for non-reasoning models against the maximum the creator discloses for reasoning models, and up to 30 automatic retries on API failures, with any question that fails all 30 reviewed by hand. Agentic benchmarks run in E2B sandboxes.^[3]

The methodology page also flags contamination risk that is outside its control. It notes that the authors of Humanity's Last Exam disclose that their curation process involved adversarially selecting questions using GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, o1, o1-mini, and o1-preview, and cautions against directly comparing those models against later ones that were not involved in curation.^[23]

Grader models and cross-version comparability

Several evaluations in the index are scored by another language model rather than by a program, which makes the grader itself a moving part of the measurement. Version 4.1.1, current from August to September 2026, changed the grader for three of them at once: Humanity's Last Exam, AA-LCR and AA-Omniscience all moved to GPT-5.6 Luna (medium), replacing GPT-4o (August 2024), Qwen3 235B A22B 2507 Non-Reasoning and Gemini 3 Flash Preview (Reasoning) respectively.^[3] Two earlier point releases were grader swaps forced by deprecations, v4.0.3 for AA-Omniscience and v4.0.4 for GDPval-AA.^[3] Artificial Analysis says outright that the AA-LCR v1.1 upgrade, which added a grading system prompt, corrected 16 answer keys and changed grader, means "Scores are not directly comparable with v1.0."^[3] The same caution applies between two organizations running the same benchmark. On GDP.pdf, where Artificial Analysis judges with GPT-5.6 Luna Medium and Surge AI uses Gemini 3.5 Flash, the methodology page states: "The task set is shared, but the document input and judge differ, so Artificial Analysis and Surge scores are not directly comparable."^[3]

The practical consequence is that a model's index score can move without the model changing. MiniCPM5-2B, released by OpenBMB on 7 September 2026, scored 23 on v4.1.1 and 15 on v4.2, and its model page showed 14 on v4.3 as of 8 September 2026, marked as an estimate pending independent evaluation.^[33][38] Artificial Analysis attributed the drop to "the updated evaluation mix and weightings" and stated that "scores across the two versions are not directly comparable."^[38] Comparisons between models therefore have to be drawn inside a single index version, and the site labels every Intelligence Index chart with the version that produced it.^[33]

Reasoning Model Handling

For reasoning models that generate internal "thinking" tokens before producing a visible answer, the platform assumes 2,000 reasoning tokens when the actual count is not available from the provider.^[3] This estimate is derived from testing across 60 prompts covering diverse topics including math, coding, and science.^[3]

AA-Omniscience: Knowledge and Hallucination Benchmark

AA-Omniscience is a proprietary benchmark developed by Artificial Analysis to measure factual recall and knowledge calibration.^[12] It consists of 6,000 questions derived from authoritative academic and industry sources, spanning 42 economically relevant topics across six domains:^[12]

The AA-Omniscience Index uses a scoring system bounded between -100 and +100. A correct answer earns +1, an incorrect answer costs -1, and abstaining from answering earns 0.^[12] This design rewards models that recognize their own limitations and refuse to answer rather than hallucinate. A score of 0 means the model answers correctly as often as it answers incorrectly.^[12]

What did AA-Omniscience find about hallucination?

A key finding from AA-Omniscience is that all but three of the evaluated models are more likely to hallucinate than provide a correct answer when faced with difficult questions: only three models scored above zero, with Claude 4.1 Opus achieving the highest index score of 4.8.^[12] The paper concludes that "these results reveal persistent factuality and calibration weaknesses across frontier models."^[12] The benchmark was published as a research paper on arXiv (2511.13029) in November 2025, and the dataset is publicly available on Hugging Face.^[12]

AA-Briefcase: Long-Horizon Knowledge Work

AA-Briefcase, released on 18 June 2026, is a proprietary evaluation of how well models handle realistic, long-horizon office work rather than single-turn questions.^[24] Each scenario is a multi-week project made up of linked tasks and thousands of input source files, and each task requires the model to produce a deliverable such as a spreadsheet model, a slide deck, or a memo.^[24] It ran as a standalone evaluation until it entered the Intelligence Index in v4.2 on 4 September 2026, weighted at 15%, which ties AA-Omniscience for the heaviest weight on any single evaluation in v4.3. Because its four scored scenarios are private, it also carries much of the increase in held-out weighting that Artificial Analysis gave as the reason for the update.^[3][32] The methodology page describes the same 91 tasks across four scenarios that the June announcement did.^[3]

The headline result at release was how far from saturation the benchmark is. The leading model satisfied every rubric check on only 3% of tasks, and on 31 of the 91 tasks no model exceeded a 50% rubric pass rate. Measured cost per task varied by more than 800x across the models tested, from about $0.04 to over $31.^[24]

Models run through Stirrup in a week-scoped E2B sandbox with no internet access, are capped at 500 turns per task, and complete each task in an independent run without carrying over their own earlier submissions. Rubric grading and the analytical-quality and presentation comparisons use a three-judge panel (Claude Opus 4.8 at max effort, GPT-5.5 at high reasoning and Gemini 3.1 Pro Preview at high reasoning) rather than a single judge, which Artificial Analysis says reduces bias toward submissions from the same model or model family.^[3]

Evaluations outside the Intelligence Index

Artificial Analysis runs a second tier of evaluations covering multilingual, visual, mathematical and domain-specific ability. These are reported separately and do not contribute to the Intelligence Index score.^[3]

The Artificial Analysis Multilingual Index is built on Global-MMLU-Lite and covers sixteen languages: English, Chinese, Hindi, Spanish, French, Arabic, Bangla, Portuguese, Indonesian, Japanese, Swahili, German, Korean, Italian, Yoruba and Burmese.^[3] MLCR-AA feeds a separate Healthcare and Medical Index rather than the Intelligence Index, and AA-AnalystAgent is reported as its own leaderboard.^[3]

Openness Index

The Artificial Analysis Openness Index is a standardized, independently assessed measure of how "open" AI models are. The index evaluates models across two dimensions: availability and transparency.^[11]

Each component is scored on a 0-3 qualitative scale based on best-fitting openness archetypes, with data elements averaged between pre-training and post-training phases.^[11] All component scores are summed (up to a maximum of 18 raw points) and normalized to a 0-100 scale.^[11] The index recognizes that "openness" in AI encompasses more than just the ability to download model weights; it also includes licensing terms, data transparency, and methodological disclosure.^[11]

Image Generation Leaderboard

Artificial Analysis operates a Text-to-Image Arena where users compare pairs of images generated from the same prompt without knowing which model produced each image.^[6] Models are ranked using an Elo rating system derived from these blind comparisons.^[6] Higher Elo scores indicate that a model is preferred more often by human evaluators.

The image generation leaderboard also tracks generation speed and pricing across providers.^[6] In addition to text-to-image generation, the platform maintains an image editing leaderboard that evaluates models on their ability to modify existing images based on text instructions.^[6]

The Elo scores are informed by tens of thousands of human image preferences. The methodology applies a linear regression model similar to how LMSYS calculates Elo scores for Chatbot Arena.^[6] The Image Arena Leaderboard is also published as a Hugging Face Space for broader accessibility.^[6]

Video Generation Leaderboard

The video generation section includes both text-to-video and image-to-video leaderboards.^[7] Models are ranked through the same blind comparison methodology used for image generation: users compare two videos generated from the same prompt without knowing which model created each video, and an Elo rating is computed from the accumulated votes.^[7]

The platform tracks video generation models both with and without audio output, maintaining separate rankings for each category.^[7] The leaderboard also highlights open-weight models separately, allowing developers to evaluate self-hostable alternatives to proprietary services.^[7]

Text-to-Speech Leaderboard

The Text-to-Speech (TTS) Arena uses a similar blind comparison methodology. Users listen to pairs of speech samples generated from the same text and select which sounds more natural.^[8] The resulting Elo ratings rank models across naturalness and quality.^[8]

The TTS leaderboard supports filtering by use case (knowledge sharing, assistants, entertainment, customer service) and accent preference (US, UK).^[8] It tracks both proprietary and open-weight models, with pricing data included for cost comparisons.^[8]

Speech-to-Text Leaderboard

For speech recognition, Artificial Analysis developed its own accuracy metric called AA-WER (Artificial Analysis Word Error Rate).^[9] The platform evaluates dozens of speech-to-text models and ranks them by transcription accuracy.^[9] The leaderboard distinguishes between proprietary and open-weight models, providing developers with options for both cloud-hosted and self-hosted deployments.^[9]

Speech-to-Speech Leaderboard

Artificial Analysis runs a separate program for native speech-to-speech models, meaning models that accept audio input and generate audio output without an intermediate text step. Its methodology page describes the coverage as "four complementary dimensions: speech reasoning, conversational dynamics, agentic performance and human preference in live conversations."^[40] The leaderboard adds latency and price measurements on top of those, and it also carries a small number of "Default Cascaded System" entries, which Artificial Analysis defines as a provider's own standard voice AI pipeline wiring a speech recognition model, an LLM and a text-to-speech model together, "rather than a custom pipeline optimized by Artificial Analysis." When the leaderboard was read on September 11, 2026 it carried three scored entries in that category: Deepgram Voice Agent, ElevenLabs Agents and Inworld Realtime, from Deepgram, ElevenLabs and Inworld AI. A tool-registration table printed beneath the same definition, dated "As of July 2026", documents a fourth harness, Cartesia Line, which does not carry a scored entry.^[41]

The Artificial Analysis Speech to Speech Index

The composite score is the Artificial Analysis Speech to Speech Index. The methodology page describes the current version as "an equal-weighted score for native audio models" combining Speech Reasoning (Big Bench Audio), Agentic Performance (tau-Voice), Arena Preference (Arena Score) and Task Success Rate at 25% each, and states that "Models must have valid results for all four components to appear in the index."^[40] The leaderboard repeats the same definition and labels its index chart "Only models with all data available shown."^[41]

That eligibility rule bites in practice. Reading the leaderboard's summary table on 11 September 2026, 14 of the listed entries carried an index score; the rest were missing at least one component. Both OpenAI GPT-Live-1 configurations were in that second group, with published speech reasoning, conversational dynamics and agentic performance results but no Speech Agent Arena data and therefore no index score.^[41]

Version history of the Speech to Speech Index

Artificial Analysis publishes a dated changelog for this index as it does for the Intelligence Index. The index has changed composition twice since launch, and the two changes happened in the same month.^[40]

The changelog entry for v2.0 reads: "Replaced Conversational Dynamics (Full Duplex Bench) with Task Success Rate in the index."^[40] Because Arena Preference and Task Success Rate both come out of the Speech Agent Arena, half of the current index weight now rests on that one evaluation.

Dropping Conversational Dynamics from the index did not retire the benchmark. It remains a published section of the leaderboard with its own chart, its own per-metric category breakdown and its own model coverage, and it sits in the same summary table as the index components.^[41] The distinction matters for how the measure gets cited elsewhere. When OpenAI announced GPT-Live-1 in its API on 10 September 2026, one of the benchmark charts in the post was headed "Artificial Analysis Conversational Dynamics", a measure Artificial Analysis had taken out of its index in August 2026 but still published in its own right.^[41][45]

Speech Reasoning: Big Bench Audio

Speech Reasoning is measured with Big Bench Audio, a dataset Artificial Analysis released on 20 December 2024 and published on Hugging Face under an MIT license.^[44] It contains 1,000 audio questions adapted from four categories of BIG-Bench Hard, 250 each: formal fallacies, navigate, object counting and web of lies. The four were chosen, the release post says, because they avoid tasks that depend on visual elements or that become ambiguous when read aloud. The audio was generated with 23 synthetic voices from top-ranked models in the platform's own Text-to-Speech Arena, and each generation was checked by Levenshtein distance against a transcription.^[40][44]

Models are given an audio file and must produce audio output. A judge model receives the candidate answer, the official answer and the original question, and labels the candidate correct or incorrect.^[40] In the setup described at release, audio answers were first transcribed to text with OpenAI's Whisper API.^[44] The release post framed the dataset around a gap between reading and listening: GPT-4o (August 2024) scored 92% on the text-to-text version of the questions, while its speech-to-speech counterpart, GPT-4o Realtime Preview (October 2024), reached 66%, and a cascaded Whisper plus GPT-4o plus TTS-1 pipeline lost very little accuracy relative to plain text.^[44]

The judge and the scoring rule have both changed since, which makes Big Bench Audio scores version-dependent in the same way Intelligence Index scores are.^[40]

Conversational Dynamics

Conversational Dynamics is Artificial Analysis' own implementation of a subset of Full Duplex Bench v1 and v1.5, academic benchmarks for the interactive behavior of full-duplex spoken dialogue systems. The leaderboard describes the score as a "Weighted average of pause handling, turn-taking, interruption handling, and backchannel handling from Full Duplex Bench v1 and v1.5."^[40][41]

The board publishes both the combined score and a per-metric breakdown, and its FAQ stated that conversational dynamics had been measured across 30 models when read on 11 September 2026.^[41]

Agentic Performance: tau-Voice

Agentic Performance is Artificial Analysis' implementation of tau-Voice, the voice extension of the tau-bench line from Sierra, and asks whether a model can carry a simulated customer through a complete support interaction. The tested model is prompted as a customer support agent with domain tools and a policy document; each scenario has one valid database end-state, and grading compares the final state against that ground truth. The metric is task completion pass@1, and the methodology says each score is the mean of three independent trials.^[40]

The simulated customer is itself synthetic. Customer voices are generated with ElevenLabs from prompts adapted from Sierra's public tau-Voice implementation, using what Artificial Analysis calls two control personas representing standard English speakers and five regular personas representing diverse accents and speaker profiles. The regular set ranges from an elderly caller who normally has a relative handle the task to a hurried Senegalese caller speaking English with a French accent, and the methodology page publishes the voice-generation prompt for each one.^[40]

Speech Agent Arena

The Speech Agent Arena, announced on 24 August 2026, is the newest component and the only one with humans in the loop. Artificial Analysis describes it as "a blind preference benchmark that evaluates which native audio model participants prefer in live voice conversations."^[40][42] Each round gives one participant a scenario, has them complete it separately in live voice calls with two hidden models, and then records a forced overall preference; diagnostic questions are recorded separately from that vote. The scenario pool is 35 scenarios: 15 agentic ones where the model is given tool schemas, and 20 non-agentic ones it must handle from its system prompt alone.^[40][43]

Two published measures come out of the arena, and both are index components.

Preference Elo is fitted from the pairwise votes by Bradley-Terry maximum likelihood, with GPT Realtime 1.5 anchored at 1000; approximate 95% confidence intervals use the same Hessian-based method as the Text-to-Speech Arena. For index use, the Elo is converted to an Arena Score on a 0-100 scale against an 800-Elo baseline by the formula Arena Score = 100 / (1 + 10^((800 - frozen Elo) / 400)). The methodology states that the rating is frozen "at the point the model became eligible for publication", so the index component deliberately does not track the live Elo shown on the arena charts.^[40][41]

Task Success Rate is the share of eligible conversations in which the model made the correct final task-completing tool call or calls. Two separate LLM judges run in sequence: the first decides whether the participant actually attempted the assigned task and gave the model a fair opportunity to act, and only eligible conversations pass to the second, which checks the recorded tool trace for the required final calls and their arguments. Participant deviations and unverifiable conversations are excluded before the rate is calculated, and a 95% confidence interval is reported using the Wilson score interval. Artificial Analysis publishes both judge prompts in full. Its publication rule is that "Overall results are published for a model with at least 100 appearances and a 95% confidence interval half-width no greater than 75," and participant recordings and transcripts are not published.^[40]

Participants are drawn from a "qualified pool of paid, screened third-party participants", and apart from one worked example built around a new-patient dental booking, the scenario prompts, tool schemas and participant instructions are "currently private to reduce overfitting."^[42][43]

The launch results made the case for measuring preference and task success separately. Gemini 3.1 Flash Live Minimal led overall preference at 1,046 Elo but recorded a 74.6% Task Success Rate, while Grok Voice Think Fast 2.0 High led task success at 94.7%. Artificial Analysis wrote that "a preferred conversation does not always result in successful task completion", noting that a conversation can sound as though the requested action went through even when the required final tool call did not.^[42]

Speed and price

Time to First Audio (TTFA) is the average number of seconds to generate the first token of audio output, measured across the Big Bench Audio question set; Artificial Analysis calls it "a critical indicator of perceived responsiveness in voice agent applications."^[40] Cost per Hour of Input Audio is the cost of completing a fixed 40-question Big Bench Audio subset, normalized to an hourly basis by input audio duration. It counts audio input, audio output, text input, text output and separately exposed reasoning tokens, and excludes cached-token discounts and tool-call costs.^[41] List prices per hour of audio input and of audio output are published alongside it.^[40]

Leaderboard results, September 2026

The entries below are the ones carrying a Speech to Speech Index score in the leaderboard's summary table, read on 11 September 2026. Artificial Analysis revises these boards continuously and without notice, so the figures are a snapshot rather than a standing result. Percentages are reproduced at the precision the table displays.^[41]

Two features of the table are worth reading carefully. The index ordering and the arena ordering are not the same: the two Gemini 3.1 Flash Live entries held the two highest Preference Elo ratings in the table while placing fifth and eleventh on the index, which gives Arena Preference one quarter of the weight. And a high component score does not carry a model on its own, as Qwen Audio 3.0 Realtime Plus led both speech reasoning and conversational dynamics yet sat ninth on the index, held down by the lowest Preference Elo of any entry with an index score.^[41]

Hardware Benchmarking

Beyond software models and API providers, Artificial Analysis provides benchmarking of AI accelerator hardware for inference workloads.^[10] The platform measures how performance scales with concurrent load across different GPU systems, including NVIDIA H100, H200, and B200 configurations, as well as AMD MI300X and Google TPU v6e (Trillium) chips.^[10]

The cost per million tokens is calculated by combining system output throughput with the average cloud price per GPU per hour.^[10] Hardware benchmarks are conducted periodically, at least once per quarter, with full specifications published alongside results.^[10]

AA-AgentPerf

On 12 June 2026 Artificial Analysis published the first results from AA-AgentPerf, a hardware benchmark built around agentic rather than single-turn workloads.^[25] Instead of issuing synthetic prompts, AA-AgentPerf replays recorded coding-agent sessions that resolve real repository issues, running up to 200 turns per session with input sequences ranging from roughly 5,000 to about 131,000 tokens (mean around 27,000) and a median output of about 150 tokens per turn.^[25]

The benchmark asks how many concurrent agents a system can serve while still meeting market-derived service-level objectives for output speed and time to first token, and its headline metric is agents per megawatt: the maximum number of agents an accelerator platform can serve for each megawatt of power drawn.^[25] Unlike the platform's earlier hardware tests, AA-AgentPerf permits the optimizations that inference providers actually run in production, including KV cache reuse, speculative decoding, and disaggregated prefill and decode.^[25]

The first round covered NVIDIA Blackwell systems (GB300 NVL72 and eight-way B300), eight-way H200 Hopper systems, and eight-way AMD MI355X systems, all serving DeepSeek V4 Pro. Artificial Analysis reported that rack-scale deployments were more efficient than single accelerators on this workload and that Blackwell represented a large generational step over Hopper in concurrent-agent capacity.^[25] The hardware section also lists a workstation tier and a portable-device tier as planned additions during 2026.^[10]

Stirrup: Open-Source Agent Framework

In December 2025, Artificial Analysis released Stirrup, an open-source lightweight framework for building AI agents.^[13] The framework was developed as part of the team's work on evaluating agentic capabilities through benchmarks like GDPval-AA.^[13]

Stirrup differs from many existing agent frameworks by letting models drive their own workflow rather than imposing rigid step-by-step processes.^[13] The framework provides models with essential tools including code execution environments, web search, web browsing, and bash command execution in a sandboxed environment.^[13] The design philosophy draws from analysis of leading agents such as Claude Code and Codex.^[13] Stirrup is distributed as a Python package installable with "pip install stirrup," ships a TypeScript implementation called StirrupJS, and is available on GitHub under the Artificial Analysis organization with features like context management and MCP (Model Context Protocol) support.^[13][20]

Hugging Face Integration

Artificial Analysis publishes several of its leaderboards as Hugging Face Spaces, making them accessible to the broader AI research community.^[15] The LLM Performance Leaderboard was brought to Hugging Face in May 2024 as a collaborative effort.^[15] The Text-to-Image Arena Leaderboard and Text-to-Video Arena Leaderboard are also available as Hugging Face Spaces.^[6][7]

This integration gives researchers and developers access to the same data through the Hugging Face ecosystem, where it can be referenced alongside model cards, datasets, and other evaluation tools.

API and Data Access

Artificial Analysis provides a free public API that gives developers programmatic access to its benchmark data.^[14] The API covers model intelligence evaluations, speed benchmarks, pricing data, and Elo ratings across different model categories.^[14] The free tier is rate-limited to 1,000 requests per day.^[14]

The website and all public leaderboards are freely accessible without an account.^[1] Enterprise customers who require more detailed analysis, custom evaluations, or standardized reports subscribe to the Artificial Analysis Insights Platform.^[17]

Revenue Model

How does Artificial Analysis make money?

Artificial Analysis generates revenue through two primary streams. The first is enterprise subscriptions, which provide standardized reports on model deployment decisions covering topics like serverless versus managed infrastructure versus leasing chips.^[17] The second is private benchmarking, where AI companies commission custom evaluations of their models.^[17] The founders have emphasized that no company pays to appear on the public website, maintaining the platform's independence; as Cameron put it, "you can't pay us for better results."^[16][17]

Industry Recognition and Adoption

The Stanford AI Index Report 2025 cited Artificial Analysis benchmarks as key reference data for understanding the AI model landscape. Academic papers on LLM pricing and performance have also referenced the platform's data.^[19] As of November 2025, the site recorded 27.56% month-over-month traffic growth, reflecting increasing adoption among developers and decision-makers.^[17]

Groq, the inference chip company, has publicly highlighted its performance on Artificial Analysis leaderboards, demonstrating the platform's influence on how AI companies market their products.^[18] Multiple cloud providers and inference startups reference their Artificial Analysis rankings in marketing materials and press releases.^[18]

Comparison with Other Benchmarking Platforms

How does Artificial Analysis differ from Chatbot Arena?

Artificial Analysis occupies a distinct niche in the AI evaluation ecosystem. While platforms like LMSYS Chatbot Arena focus on human preference rankings through pairwise comparisons, and the Open LLM Leaderboard on Hugging Face focuses on academic benchmarks for open models, Artificial Analysis combines quality evaluation with real-world performance and pricing data.^[1]

The platform's breadth across multiple modalities (text, image, video, speech) and its focus on practical metrics like pricing and provider-level performance differentiate it from purely academic benchmarking efforts.^[1]

How does Artificial Analysis differ from MLPerf and InferenceX?

The three most prominent sources of public inference-performance numbers in 2026 differ less in what they measure than in who produces the numbers and how often.

MLPerf, run by the nonprofit consortium MLCommons, is submission-based. MLCommons defines the reference implementations, datasets and quality targets; vendors, cloud providers and integrators run them on hardware they control and submit results in fixed rounds roughly six months apart; submitters review each other's entries before publication, and each published number is tied to a disclosed system configuration. MLCommons describes the goal as measuring system performance in an "architecture-neutral, representative, and reproducible" way. MLPerf Inference v6.0 was published in April 2026 and MLPerf Training v6.0 in June 2026.^[26][27]

InferenceX, the open-source benchmark published by SemiAnalysis, is continuously re-run. Launched in October 2025 as InferenceMAX and renamed InferenceX at its v2 release on 16 February 2026, it commits its benchmark recipes to a public repository and runs them as GitHub Actions workflow jobs on a fleet of NVIDIA and AMD accelerators, so every point on its dashboard links back to the public CI run behind it. It benchmarks open-source serving stacks including vLLM, SGLang and TensorRT-LLM, and reports throughput per GPU against interactivity, tokens per megawatt, cost per million tokens and joules per token. Its stated motivation is that "benchmarks conducted at a fixed point in time quickly go stale and do not represent the performance that can be achieved with the latest software packages."^[28][29][30]

Artificial Analysis sits between the two. Like InferenceX it publishes results it generates itself rather than results submitted by vendors, and it updates them continuously rather than in rounds; unlike InferenceX its evaluation harnesses, proprietary datasets such as AA-Omniscience and AA-Briefcase, and much of its raw data are not open source, and its coverage extends to model quality and to commercial API endpoints rather than to self-hosted serving stacks alone.^[2][24] The three overlap most directly on accelerator benchmarking, where AA-AgentPerf, MLPerf Inference and InferenceX all report per-system inference performance under different rules about which production optimizations are allowed.^[25][26][29]

See Also

• WeirdML
• Large Language Models
• Chatbot Arena
• MMLU
• Hugging Face
• Prompt Engineering
• AI model comparison finder

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Artificial Analysis. "AI Model & API Providers Analysis." artificialanalysis.ai. Accessed June 2026.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13Artificial Analysis. "Language Model Benchmarking Methodology." artificialanalysis.ai/methodology. Accessed June 2026.
3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36 ^37 ^38 ^39Artificial Analysis. "Intelligence Benchmarking Methodology (Intelligence Index v4.3)." artificialanalysis.ai/...intelligence-benchmarking. Accessed September 8, 2026.
4. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "LLM Leaderboard." artificialanalysis.ai/leaderboards/models. Accessed June 2026.
5. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "LLM API Providers Leaderboard." artificialanalysis.ai/leaderboards/providers. Accessed June 2026.
6. ^1 ^2 ^3 ^4 ^5 ^6 ^7Artificial Analysis. "Text to Image Leaderboard." artificialanalysis.ai/image/leaderboard/text-to-image. Accessed June 2026.
7. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "Text to Video Leaderboard." artificialanalysis.ai/video/leaderboard/text-to-video. Accessed June 2026.
8. ^1 ^2 ^3 ^4Artificial Analysis. "Text to Speech Leaderboard." artificialanalysis.ai/text-to-speech/leaderboard. Accessed June 2026.
9. ^1 ^2 ^3Artificial Analysis. "Speech to Text (ASR) Providers Leaderboard." artificialanalysis.ai/speech-to-text. Accessed June 2026.
10. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "AI Hardware Benchmarking & Performance Analysis." artificialanalysis.ai/benchmarks/hardware. Accessed August 1, 2026.
11. ^1 ^2 ^3 ^4Artificial Analysis. "Introducing the Artificial Analysis Openness Index." artificialanalysis.ai/articles/announcing-artificial-analysis-openness-index. Accessed June 2026.
12. ^1 ^2 ^3 ^4 ^5 ^6 ^7Artificial Analysis. "AA-Omniscience: Knowledge and Hallucination Benchmark." artificialanalysis.ai/evaluations/omniscience; arXiv:2511.13029. November 2025.
13. ^1 ^2 ^3 ^4 ^5 ^6Artificial Analysis. "Stirrup: Our new open source framework for building agents." artificialanalysis.ai/articles/stirrup-open-source-framework-agents. December 2025.
14. ^1 ^2 ^3Artificial Analysis. "API Documentation." artificialanalysis.ai/documentation. Accessed June 2026.
15. ^1 ^2Hugging Face. "Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face." huggingface.co/blog/leaderboard-artificial-analysis. May 2024.
16. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13Latent Space. "Artificial Analysis: Independent LLM Evals as a Service with George Cameron and Micah Hill-Smith." latent.space/p/artificialanalysis. Accessed June 2026.
17. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9The Neuron. "The State of AI in November 2025: A Deep Dive with the Co-Founders of Artificial Analysis." theneuron.ai. November 2025.
18. ^1 ^2Groq. "Groq LPU Tops Latency & Throughput in Benchmark." groq.com/blog/artificialanalysis-ai-llm-benchmark. Accessed June 2026.
19. ^TechCrunch. "The rise of AI reasoning models is making benchmarking more expensive." techcrunch.com. April 2025.
20. ^GitHub. "ArtificialAnalysis/Stirrup." github.com/ArtificialAnalysis/Stirrup. Accessed June 2026.
21. ^Artificial Analysis. "Intelligence Index Evaluation: GPQA Diamond." artificialanalysis.ai/evaluations/artificial-analysis-intelligence-index. Accessed June 2026.
22. ^1 ^2Artificial Analysis. "AI Model & API Providers Analysis" (homepage model and provider counts). artificialanalysis.ai. Accessed September 8, 2026.
23. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "Methodology" (blended price, cost per task, tokenizer standardization) and "Intelligence Benchmarking Methodology." artificialanalysis.ai/methodology; artificialanalysis.ai/...intelligence-benchmarking. Accessed August 1, 2026.
24. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Artificial Analysis. "Announcing AA-Briefcase: a frontier knowledge work evaluation." artificialanalysis.ai/...aa-briefcase. June 18, 2026.
25. ^1 ^2 ^3 ^4 ^5 ^6Artificial Analysis. "First results from AA-AgentPerf: the hardware benchmark for the agent era." artificialanalysis.ai/...aa-agentperf. June 12, 2026.
26. ^1 ^2 ^3MLCommons. "MLCommons Releases New MLPerf Inference v6.0 Benchmark Results." mlcommons.org/...mlperf-inference-v6-0-results. April 1, 2026.
27. ^MLCommons. "MLPerf Training v6.0 Results: New MoE Benchmarks and Record System Diversity." mlcommons.org/...mlperf-training-v6-0-results. June 16, 2026.
28. ^1 ^2SemiAnalysis. "InferenceMAX: Open Source Inference Benchmarking." newsletter.semianalysis.com/...en-source-inference. October 9, 2025.
29. ^1 ^2 ^3InferenceX by SemiAnalysis. "About." inferencex.semianalysis.com/about. Accessed August 1, 2026.
30. ^SemiAnalysis. "InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper (Formerly InferenceMAX)." newsletter.semianalysis.com/...nvidia-blackwell-vs. February 16, 2026.
31. ^1 ^2 ^3 ^4 ^5 ^6Artificial Analysis. "Announcing the Artificial Analysis Intelligence Index v4.3." artificialanalysis.ai/...s-intelligence-index-v4-3. September 7, 2026.
32. ^1 ^2 ^3 ^4 ^5 ^6 ^7Artificial Analysis. "Announcing Artificial Analysis Intelligence Index v4.2." artificialanalysis.ai/...s-intelligence-index-v4-2. September 4, 2026.
33. ^1 ^2 ^3 ^4 ^5Artificial Analysis. "MiniCPM5-2B: Intelligence, Performance & Price Analysis" (model class definitions, price bands, estimate labelling, Intelligence Index v4.3 score). artificialanalysis.ai/...minicpm5-2b. Accessed September 8, 2026.
34. ^Zapier. "AutomationBench" repository README (public and private task sets, domain coverage). github.com/...AutomationBench. Accessed September 8, 2026.
35. ^"AutomationBench." arXiv:2604.18934. April 21, 2026.
36. ^"GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents." arXiv:2607.11192. July 2026.
37. ^Terminal-Bench. "Terminal-Bench 4.0" leaderboard, hosted by Stanford, Harbor and the Laude Institute. tbench.ai. Accessed September 8, 2026.
38. ^1 ^2 ^3Artificial Analysis. "OpenBMB releases MiniCPM5-2B." artificialanalysis.ai/...nbmb-releases-minicpm5-2b. September 7, 2026.
39. ^"SciCode: A Research Coding Benchmark Curated by Scientists." arXiv:2407.13168. July 2024.
40. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16Artificial Analysis. "Speech to Speech Benchmarking Methodology" (Speech to Speech Index composition and version history, Big Bench Audio, Conversational Dynamics, tau-Voice, Speech Agent Arena, price and speed definitions). artificialanalysis.ai/...ch-to-speech-benchmarking. Accessed September 11, 2026.
41. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11Artificial Analysis. "Speech to Speech AI Model & Provider Leaderboard" (index chart and definition, per-benchmark charts, summary table of key metrics, cascaded-system entries, FAQ). artificialanalysis.ai/speech-to-speech. Accessed September 11, 2026.
42. ^1 ^2 ^3Artificial Analysis. "Announcing the Speech Agent Arena: Compare Speech agents in real world conversations." artificialanalysis.ai/...ng-the-speech-agent-arena. August 24, 2026.
43. ^1 ^2Artificial Analysis. "Speech Agent Arena Overview." artificialanalysis.ai/...overview. Accessed September 11, 2026.
44. ^1 ^2 ^3 ^4Micah Hill-Smith and George Cameron. "Evaluating Audio Reasoning with Big Bench Audio." Hugging Face blog, December 20, 2024. huggingface.co/...big-bench-audio-release. Dataset: huggingface.co/...big_bench_audio
45. ^OpenAI. "Build more natural voice experiences with GPT-Live-1 in the API." September 10, 2026. openai.com/...introducing-gpt-live-1-in-the-api (chart titles read from the page's embedded chart data, via Internet Archive captures of September 10 and 11, 2026).