# Task-completion time horizon (METR)

> Source: https://aiwiki.ai/wiki/metr_time_horizon
> Updated: 2026-09-15
> Fact-checked: 2026-09-15
> Categories: AI Benchmarks, AI Safety, Model Evaluation
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Task-completion time horizon (METR)." aiwiki.ai, 15 Sept 2026. https://aiwiki.ai/wiki/metr_time_horizon
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**The task-completion time horizon** is a metric for AI capability proposed by [METR](https://aiwiki.ai/wiki/metr) that expresses a model's ability in units of human time: it is the length of task, measured by how long a human expert takes to do it, that the model can finish with a given success rate. The headline version is the 50%-task-completion time horizon, the human task length at which a model succeeds half the time. It was introduced in a March 2025 paper by Thomas Kwa, Ben West and colleagues at METR, published as arXiv:2503.14499 under the title "Measuring AI Ability to Complete Long Tasks" and retitled "Measuring AI Ability to Complete Long Software Tasks" in a later revision. The metric is best known for an empirical finding: on a suite of software and research tasks, the time horizon of [frontier models](https://aiwiki.ai/wiki/frontier_models) roughly doubled every seven months from 2019 through early 2025. [1][2]

The appeal of the metric is that it puts a single, interpretable number on otherwise hard-to-compare systems. Saying a model scores 71% on some [benchmark](https://aiwiki.ai/wiki/benchmark) tells you little about what it can actually do. Saying it can complete tasks that take a human about an hour, with even odds, is something a non-specialist can reason about, and it ties model progress to a quantity (human labor time) that matters for forecasting economic and safety impacts.

That interpretability is also the metric's main hazard. METR has published a running series of notes correcting misreadings of its own chart, revised the task suite and the evaluation stack in January 2026, and fixed a modelling error in March 2026 that moved published numbers. Any specific time-horizon figure therefore needs four qualifiers to be meaningful: which version of the estimate (TH1.0 or TH1.1), which task suite, which success threshold, and as of which date. [4][6][15]

## What the metric measures

Most AI evaluations report an average score on a fixed set of problems. That number drifts as benchmarks saturate, and it does not translate across tasks of different difficulty. METR's idea is to flip the framing. Instead of asking what fraction of tasks a model solves, ask: how hard a task, in human terms, can the model handle before its reliability drops to a chosen threshold? Difficulty is operationalized as the time a skilled human needs to complete the task. [1]

The result is a measure on a meaningful axis. A model with a 5-minute horizon can knock out quick scripted jobs but falls apart on anything longer. A model with a 1-hour horizon can carry out tasks with many steps that a person would budget real time for. Because the unit is human time rather than a percentage, horizons from different model generations and different task mixes are at least roughly comparable, and the trend over time becomes legible. [2]

METR built the metric mainly around autonomous software engineering and machine-learning research work, the kind of activity [AI agents](https://aiwiki.ai/wiki/ai_agents) attempt when given a coding environment, a shell, and a goal. The motivation is partly practical and partly about [AI safety](https://aiwiki.ai/wiki/ai_safety): long-horizon autonomy is one of the capabilities that would make a system genuinely able to act in the world without close supervision, so tracking it is useful for risk assessment as well as for capability forecasting. [1][2]

What the number is not is the length of time an agent can run unsupervised. METR's FAQ states this plainly: the 50% time horizon "is a measure of the difficulty of a task, rather than the time an AI spends to complete the task", and agents that succeed are typically several times faster than the humans whose times set the scale. [4] Thomas Kwa, the paper's lead author, put the same point first in his January 2026 note on the metric's limitations: time horizon is "the amount of serial human labor they can replace with a 50% success rate", not a duration of autonomous operation. [7]

METR also warns that the human baseline is a low-context one. The contracted baseliners are skilled professionals in software engineering, machine learning or cybersecurity with an average of about five years of experience, but they come to each task cold. METR's own framing is that a two-hour task should be read as what "someone with low or no prior context (like a new hire or freelance contractor) could complete in 2 hours", not what an experienced engineer already fluent in a codebase could do. [4]

## The task suites

The original study used a combined collection of 170 tasks that METR calls METR-HRS, drawn from three sources. [3]

HCAST (Human-Calibrated Autonomy Software Tasks) supplies 97 tasks spanning software engineering, [cybersecurity](https://aiwiki.ai/wiki/ai_in_cybersecurity), [machine learning](https://aiwiki.ai/wiki/machine_learning), and general reasoning, described in the paper as ranging from 1 minute to around 30 hours of human time. [RE-Bench](https://aiwiki.ai/wiki/re_bench) contributes 7 difficult machine-learning research engineering tasks, all eight hours long. SWAA (Software Atomic Actions) is a set of 66 single-step tasks that METR built specifically for this work, ranging from roughly 1 second to 30 seconds. METR's later work describes the combined suite as spanning about 1 second to 16 hours. [3][9]

The short SWAA tasks matter for one reason: without them, the easiest tasks took humans about a minute, and early models could not register on that scale at all. Adding second-scale tasks let METR place models as old as [GPT-2](https://aiwiki.ai/wiki/gpt-2) and GPT-3 on the same curve as 2025 systems, which is what makes a multi-year trend possible. [2][3]

The human side of the comparison is not estimated casually. METR reports over 800 human baseline runs totaling 2,529 hours of work, performed by contractors and staff with relevant expertise, to establish how long each task actually takes a competent person. Those baselines define the time axis against which every model is scored. [3]

Tasks are grouped into families of similar problems (for example, a "crossword" family containing a 3x3 and a 5x5 puzzle), and families with many members are down-weighted so that one prolific family cannot dominate the fit. [3]

## How the horizon is estimated

For each model, METR runs it on every task with multiple attempts, typically six to eight independent runs, and records success or failure. It then fits a [logistic regression](https://aiwiki.ai/wiki/logistic_regression) that predicts the probability of success as a function of the logarithm of the human task length. The fitted curve slopes downward: models almost always succeed at very short tasks and almost always fail at very long ones. [2][3][12][15]

The model has two free parameters per agent. One, written h, is the task length at which predicted success crosses 50%, which is the 50% time horizon itself. The other, the slope beta, controls how sharply success falls off as tasks get longer, and it is what converts the 50% horizon into any other threshold. METR notes that this two-parameter form is the reason the 80% horizon is not an independent estimate: there are not enough parameters to fit the top and bottom of the success curve separately, so improving performance on tasks near the 20% horizon can mechanically lower the reported 80% horizon. [7][15]

Confidence intervals come from a hierarchical bootstrap over task families, then tasks, then individual attempts. Because the resampling is over the task set as well as the runs, METR notes that the intervals reflect the likelihood of getting the same estimate with an entirely new set of tasks, which makes them wide but also makes comparisons between models on the same tasks tighter than the individual intervals suggest. [6][15]

This methodology is related to item response theory, which models the chance that a test-taker answers an item of a given difficulty. The difference is that METR does not learn task difficulty from agent performance; it uses human baseline time as the difficulty rating. [3]

The pattern shows up cleanly in raw success rates too. METR found that current models had close to a 100% success rate on tasks taking humans less than about 4 minutes, but succeeded less than 10% of the time on tasks taking more than about 4 hours. The horizon sits in the transition zone between those regimes. [2]

### 50% versus 80%

The 50% time horizon is read off the fitted curve as the human task length at which the predicted success probability crosses one half. The same machinery yields other thresholds. The 80% time horizon, the length at which a model succeeds 80% of the time, is the figure to watch if you care about reliability rather than coin-flip odds, and it is considerably shorter. The original paper reports Claude 3.7 Sonnet at a 50% horizon of 59 minutes (rounded to "around 50 minutes" in its abstract) against an 80% horizon of about 15 minutes, and states that the 80% horizons across the studied models are roughly 5x shorter than the 50% horizons. Importantly for the trend argument, the two thresholds grew at nearly the same rate in the paper: a doubling time of 213 days at 80% against 212 days at 50%. [3]

METR does not publish horizons at higher reliability levels, and explains why. Fitting a 99% horizon would need roughly 300 highly diverse tasks in each time bucket and a label-noise rate well under 1%; with a noisier suite the benchmark saturates around 98% and every model's 99% horizon comes out as zero. [4][7]

## Time Horizon 1.1

On 29 January 2026 METR released a revised version of its estimates, labelled TH1.1, changing both the task suite and the software that runs it. [6]

The suite grew from 170 to 228 tasks. METR added 73 tasks, all HCAST tasks described in Rein (2025) that were not in the original paper but subsequently passed its quality checks; removed 15; and updated 53 (27 with a new definition, 13 with a new human time estimate, 13 with both). Removals and edits generally covered tasks whose descriptions were confusing or easy to reward-hack, or whose scoring functions had errors. The number of long tasks, estimated at 8 or more human hours, went from 14 to 31. [6]

At the same time METR moved its evaluation infrastructure from Vivaria, the runner it had built in house in 2023, to Inspect, the open-source evaluation framework developed by the [UK AI Security Institute](https://aiwiki.ai/wiki/uk_aisi). [6]

Only 14 of the 33 models with TH1 estimates were re-estimated. METR gives three reasons for the rest: the model is no longer publicly available; the model would need significant changes to the tool-calling scaffold (GPT-2, GPT-3 and GPT-3.5 are named); or the model was far from the frontier at release and so unlikely to move the trend. The new estimates "generally lie within the confidence intervals from the TH1 time horizons". [6]

The doubling time is where TH1.1 requires the most care, because METR reports two different comparisons and they do not say the same thing.

| Measure | TH1.0 | TH1.1 |
|---|---|---|
| 50% doubling time, whole period | 195.8 days, 95% CI 162-223 | 196.5 days (stitched trend) |
| 50% doubling time, 2023 onward | 165.3 days, 95% CI 129-211 | 130.8 days, 95% CI 107-161 |
| 50% doubling time, 2024 onward | 108.9 days | 88.6 days |
| Tasks | 170 | 228 |
| Tasks of 8 hours or more | 14 | 31 |
| Of those, human-baselined | 6 | 5 |
| Models estimated | 33 | 14 |
| State-of-the-art models in the fit | 17 | 11 |

Because no pre-2023 model was re-run under TH1.1, the whole-period TH1.1 figure is a stitched trend that borrows TH1 estimates for GPT-2, GPT-3 and GPT-3.5. On that basis the doubling time is "exactly the same" as under TH1, 196 days or about 7 months, although METR notes the new fit looks slightly less linear with a slightly lower R-squared. Restricted to the post-2023 period where both methods have data, the doubling time is 131 days under TH1.1 against 165 days under TH1, which METR describes as progress estimated to be 20% more rapid. The two statements are not in conflict; they are answers to different questions, and quoting either one as "the" doubling time without the qualifier is the single most common way the number gets abused. [6]

Those are the figures METR published on 29 January 2026. They were superseded five weeks later by the modelling correction described below: the live page now gives the TH1.0 whole-period doubling time as 201.2 days (95% CI 176-225) and the post-2023 figure as 175.6 days, against a TH1.1 stitched whole-period figure of 187.8 days and a post-2023 figure of 128.7 days (95% CI 104-158). The qualitative reading, that the whole-period trends agree while the post-2023 TH1.1 trend is faster, survives the correction. [4]

The shift comes from older estimates falling and newer ones rising: the two GPT-4 snapshots dropped by 35% and 57%, while GPT-5 and Claude Opus 4.5 rose by 55% and 11%. METR is candid that the TH1.1 doubling time sits inside the TH1 confidence interval, but that those intervals represent resampling an entirely new task set, whereas TH1.1 and TH1 in fact share many tasks. [6]

Doubling the number of long tasks tightened the intervals at the top of the range. The upper bound on Claude Opus 4.5 was 4.4x its point estimate under TH1 and 2.3x under TH1.1. METR still calls these intervals "very wide". It also flags a limitation that bears directly on the longest measurements: human baseline times were collected for only 5 of the 31 tasks of 8 hours or more, with the rest relying on estimates. [6]

## Measured horizons

METR maintains a live page of current estimates, which lets a reader switch between TH1.0 and TH1.1 and between the 50% and 80% thresholds. The table below gives a selection of the published models on each version, in minutes of human task time, with 95% confidence intervals on the 50% figures. The page's values incorporate the March 2026 correction described below and therefore differ from the figures printed in the original paper and in the January 2026 TH1.1 announcement. [4]

| Model | Release | TH1.0 50% | TH1.0 80% | TH1.1 50% | TH1.1 80% |
|---|---|---|---|---|---|
| GPT-2 | Feb 2019 | 0.05 [0.01, 0.14] | 0.01 | not re-estimated | not re-estimated |
| davinci-002 | May 2020 | 0.14 [0.09, 0.22] | 0.06 | not re-estimated | not re-estimated |
| GPT-3.5 Turbo Instruct | Mar 2022 | 0.6 [0.3, 1.1] | 0.26 | not re-estimated | not re-estimated |
| GPT-4 (0314) | Mar 2023 | 6.0 [2.8, 11.7] | 1.37 | 4.0 [1.9, 8.0] | 0.89 |
| GPT-4 (1106) | Nov 2023 | 9.4 [4.5, 18.2] | 2.01 | 4.0 [1.9, 8.4] | 0.78 |
| Claude 3 Opus | Mar 2024 | 7.1 [3.2, 14.8] | 1.58 | 4.0 [1.7, 8.8] | 0.64 |
| GPT-4 Turbo | Apr 2024 | 7.8 [4.1, 14.2] | 2.49 | 3.7 [2.0, 6.7] | 0.93 |
| GPT-4o | May 2024 | 10.2 [5.0, 19.5] | 2.37 | 7.0 [4.0, 12.9] | 1.27 |
| Claude 3.5 Sonnet (Jun) | Jun 2024 | 19.8 [10.5, 35.8] | 4.19 | 11.4 [5.5, 22.4] | 1.67 |
| o1-preview | Sep 2024 | 23.3 [12.6, 39.4] | 6.42 | 20.3 [11.7, 33.4] | 4.42 |
| Claude 3.5 Sonnet (Oct) | Oct 2024 | 30.5 [14.7, 60.9] | 5.91 | 20.5 [10.1, 40.8] | 2.60 |
| o1 | Dec 2024 | 41.6 [19.8, 85.8] | 7.65 | 38.8 [21.2, 65.0] | 7.09 |
| [Claude 3.7 Sonnet](https://aiwiki.ai/wiki/claude_3_7_sonnet) | Feb 2025 | 55.3 [28.9, 89.2] | 20.38 | 60.4 [33.0, 104.2] | 12.09 |
| o3 | Apr 2025 | 91.4 [46.6, 158.0] | 27.30 | 119.7 [74.6, 190.9] | 29.98 |
| [Grok 4](https://aiwiki.ai/wiki/grok_4) | Jul 2025 | 104.8 [47.6, 208.3] | 18.59 | not re-estimated | not re-estimated |
| Claude Opus 4.1 | Aug 2025 | 109.3 [56.0, 200.8] | 27.29 | 100.5 [59.3, 159.5] | 23.46 |
| GPT-5 | Aug 2025 | 130.8 [67.7, 247.8] | 32.98 | 203.0 [112.6, 405.6] | 38.31 |
| [Gemini 3 Pro](https://aiwiki.ai/wiki/gemini_3_pro) | Nov 2025 | not estimated | not estimated | 224.3 [139.6, 379.2] | 54.14 |
| [GPT-5.1-Codex-Max](https://aiwiki.ai/wiki/gpt_5_1_codex_max) | Nov 2025 | 162.3 [80.2, 336.5] | 40.04 | 223.7 [134.3, 396.2] | 50.63 |
| [Claude Opus 4.5](https://aiwiki.ai/wiki/claude_opus_4_5) | Nov 2025 | 246.8 [104.3, 821.6] | 36.04 | 293.0 [161.7, 623.7] | 49.43 |
| GPT-5.2 | Dec 2025 | not estimated | not estimated | 352.2 [198.1, 815.2] | 66.00 |
| GPT-5.3-Codex | Feb 2026 | not estimated | not estimated | 349.5 [194.9, 816.4] | 54.74 |
| [Claude Opus 4.6](https://aiwiki.ai/wiki/claude_opus_4_6) | Feb 2026 | not estimated | not estimated | 718.8 [316.7, 3633.8] | 69.87 |
| Gemini 3.1 Pro | Feb 2026 | not estimated | not estimated | 384.1 [233.5, 694.8] | 89.80 |
| GPT-5.4 | Mar 2026 | not estimated | not estimated | 341.7 [186.6, 768.8] | 53.88 |
| Claude Mythos Preview (early) | Apr 2026 | not estimated | not estimated | 1044.8 [508.9, 3304.3] | 185.91 |

The ordering is not perfectly monotonic. Chat-oriented models such as GPT-4 and Claude 3 Opus land in a similar few-minute range, while reasoning-trained models such as the o1 series jump well ahead, and later models sometimes land below an earlier one within the noise. But the long-run climb is steep: from about 3 seconds for GPT-2 in 2019 to roughly 12 hours for Claude Opus 4.6 in February 2026 and about 17 hours for an early Claude Mythos Preview measured in April 2026. [4]

That last figure is past the point METR says it can measure. When the Mythos Preview result was added on 8 May 2026, METR added a caution alongside it: "Measurements above 16 hrs are unreliable with our current task suite." It also says its coverage is not a complete record of the most capable models, and as of the page's May 2026 update listed Claude Opus 4.7, Grok 4.3 and GPT-5.5 as recent releases without a published horizon. Running one evaluation typically takes at least one to two weeks of calendar time, covering access setup, elicitation on a dev set, roughly 1,000 runs on the test set, reward-hacking review and analysis. [4]

[Epoch AI](https://aiwiki.ai/wiki/epoch_ai) mirrors the 50% series on its own benchmarks hub, sourcing the numbers directly from METR's analysis rather than re-running anything, and does not currently track the 80% series. [11]

The August 2025 GPT-5 evaluation is a useful illustration of how much the pipeline can move a single number. METR reported a 50% horizon of 2 hours 17 minutes (95% CI 65 minutes to 4 hours 25 minutes) against o3's 1 hour 30 minutes, and an 80% horizon of about 25 minutes. Rescoring 18 tasks found to contain reward hacks, about 2% of the HCAST and RE-Bench runs, had already pulled the central estimate down from roughly 3 hours, and the upper bound from 8 to 4.5 hours. Reviewing failures in the other direction, METR estimated that up to 25-35% of GPT-5's failures might be spurious, and excluding five problematic tasks raised the point estimate to 2 hours 41 minutes. [5]

## Extrapolations

The paper's own extrapolation targets a horizon of one month, which METR defines as 167 working hours rather than a calendar month, on the grounds that humans do not work around the clock. Naively extending the trend, the paper gives an 80% confidence interval for 1-month software task capability spanning late 2028 to early 2031, with a central estimate of late 2029. If progress instead follows the faster 2024-2025 trend, METR puts half the probability in 2027 and early 2028. [1][3]

The paper stresses that long extrapolations are far more sensitive to the doubling rate than to the absolute horizon. Starting from o1's 39-minute horizon and a 218-day doubling time, one-month capability is about eight doublings away, roughly 4.8 years. Doubling the doubling time pushes that out by another 4.8 years; halving the horizon estimate pushes it out by only 0.6 years. [3]

Kwa later described the two things the paper was actually trying to establish as the slope of the long-run trend, one doubling every 6-7 months, and the linear extrapolation to 167 hours around 2030, rather than the exact horizon of any particular model. [7]

## How the horizon varies across domains

In July 2025 Thomas Kwa and Vincent Cheng applied the same logistic machinery to benchmarks outside METR's own suite, to test whether the software result was an outlier. Rather than running new experiments, they selected existing benchmarks that offered some estimate of human task times plus public agent scores, fitting the horizon by maximum likelihood where per-task or per-split data existed and by assuming an average slope where only an overall score was available. [9]

| Domain | Benchmark | Task length range | Human baselines |
|---|---|---|---|
| Software development | METR-HRS | 1 s to 16 h | full |
| Software development | SWE-bench Verified | 7 min to 2 h | partial |
| Agentic computer use | OSWorld | 8 s to 45 s | full |
| Agentic computer use | WebArena | 5 s to 9 min | full |
| Math contests | Mock AIME | 5 min to 4 h | none |
| Math contests | MATH | 2 s to 30 min | partial |
| Competitive programming | LiveCodeBench | 4 min to 1.5 h | none |
| Scientific QA | GPQA Diamond | 5 min to 3 h | full |
| Video QA | Video-MME | 1 s to 1 h | none |
| Autonomous driving | Tesla FSD Tracker | not applicable | not applicable |
| Simulated robotics | RLBench | 4 s to 20 s | partial |

The headline finding is that the software result is not exceptional. Scientific QA, math contests, semi-realistic software and competitive programming all sat in a 50-200 minute band with doubling times of 2-6 months. Visual computer use, measured on OSWorld and WebArena, came in 40-100x shorter but improving at a similar rate, roughly two years behind the intellectual cluster. Self-driving, measured indirectly from Tesla FSD disengagement data, improved far more slowly, around 0.6 doublings per year, or a doubling roughly every 20 months. METR's own summary is that it observed "generally similar rates of improvement to the 7-month doubling time" of the original work, and that in no domain examined was progress clearly sub-exponential. [9]

The same study is equally useful for where the metric breaks. On video question answering, video length barely predicts difficulty: Gemini 1.5 Pro scored 81.7% on videos under 2 minutes and 67.4% on 30-to-60-minute videos, so a naive extrapolation would put its horizon above 1,000 minutes. On the freelance coding benchmark SWE-Lancer, a task's monetary value, which ought to proxy engineer-hours, did not correlate with o1's success rate. METR's conclusion is that time horizon only works where human completion time genuinely tracks difficulty, which happens when a task is multiplicative (success requires getting every step right) rather than additive or a single knowledge check. [9]

## Scaffolds

METR runs most of its time-horizon measurements with two general scaffolds of its own, ReAct and Triframe, which invites the objection that purpose-built coding harnesses would score much higher. In February 2026 Nikola Jurkovic tested that directly on the TH1.1 suite, measuring Claude Opus 4.5 under [Claude Code](https://aiwiki.ai/wiki/claude_code) against ReAct and GPT-5 under [Codex](https://aiwiki.ai/wiki/openai_codex) against Triframe. Neither difference was statistically significant: Claude Code beat ReAct for Opus 4.5 in 50.7% of bootstrap samples, and Codex beat Triframe for GPT-5 in 14.5%. Quadrupling Opus 4.5's token budget from 8 million to 32 million barely changed the result. [14]

Jurkovic's stated caveats are worth carrying: METR's wrappers around the two commercial scaffolds were rough, GPT-5-Codex might use the Codex scaffold better than plain GPT-5 does, and both products are designed for interactive use with a human checking in, which is not the setting the evaluation puts them in. [14]

## Evaluation infrastructure

The stack behind the numbers has changed twice and the three names are easy to confuse. Vivaria is METR's own evaluation runner, built in house in 2023 and used for all TH1.0 measurements. Inspect is the UK AI Security Institute's open-source evaluation framework, which METR adopted for TH1.1 in January 2026 and which supplies the evaluation primitives: tasks, solvers, scorers and sandboxes. Hawk (published as Inspect-Hawk) is METR's own open-source platform, MIT licensed and created on GitHub in March 2026, that runs Inspect evaluations at scale on cloud infrastructure, provisioning isolated Kubernetes pods, managing API credentials and storing results in a warehouse. METR describes it as "our open-source platform for running AI agent evaluations at scale, built upon Inspect AI". [6][22][23]

The switch mattered for the measurements, though less than the task changes did. METR compared five models on the TH1 tasks under both runners and found the estimates close, but two models (GPT-4o and o3) scored significantly higher under Vivaria in a paired t-test, and across tasks Vivaria gave the higher score 51 times against Inspect's 31. Re-running three models on the new suite under both systems gave almost identical horizons. METR's reading is that the two models were somewhat sensitive to scaffold and prompting, and that infrastructure accounts for a relatively small share of the TH1-to-TH1.1 change. [6]

## Modelling assumptions

On 3 March 2026 METR corrected a regularization mistake in the fitting code. The model had been maximising a penalised likelihood with a term penalising the squared slope, which made every fitted curve slightly shallower than the data warranted. The fix lowered recent models' 50% horizons by up to 20% and generally raised 80% horizons. The currently published TH1.0 figures therefore differ from the ones in the paper: Claude 3.7 Sonnet is now listed at 55 minutes at 50% and about 20 minutes at 80%, where the paper reported 59 and about 15. [4][15]

Alexander Barry's March 2026 note works through what else the modelling choices are doing. His summary table of the main uncertainty sources is unusually frank for a benchmark's own maintainers.

| Source | Scale of impact on the estimate |
|---|---|
| Task distribution | Already in the intervals; often a factor of 2 in both directions |
| Choice of success-rate curve | Up to 35% lower 50% horizons and up to 100% higher 80% horizons for recent models |
| Private versus public tasks | 50% horizons generally similar; Claude Opus 4.6 falls 40% when the roughly 15% of publicly available tasks are excluded |
| Noise in task length estimates | Possibly a 30% reduction in frontier 50% horizons, with a range from 0% to 60%; capping task lengths at 8 hours costs 10-15% |

The curve-shape problem is the sharpest. A logistic sigmoid has thin tails, so it predicts near-certain success on very short tasks; when a model occasionally fails a trivial task anyway, the fit responds by flattening the whole curve, which pushes the 50% horizon up and the 80% horizon down. Barry demonstrates this by feeding the fitter a doctored dataset in which Claude Opus 4.6 succeeds on every task under 15 minutes. Despite being strictly better than its real performance, the 50% horizon drops from 12 hours to 9 while the 80% horizon nearly doubles to over 2 hours. Across eleven alternative fits including Weibull survival curves, Cauchy sigmoids, monotone splines and isotonic regression, results for Claude Opus 4.6 varied by about 1.5x at the 50% threshold and 2x at 80%. In cross-validation, a logistic model with a single slope shared across all models fit held-out task families best, suggesting the per-model slope is not worth its noise. [15]

Barry also measured the noise in the task lengths themselves and found that 80% of individual human baseline attempts fall within about 3x of the estimated true length, and 80% of researcher estimates within 4x. Because extreme values in noisy data are exaggerated, the longest tasks are probably overestimated, which biases the frontier 50% horizon up and the 80% horizon down. His conclusion is that reasonable variations mostly reduce recent 50% horizons but leave them inside the confidence intervals, and that the task distribution, not the analysis, remains the dominant uncertainty. [15]

## Caveats and limitations

METR is direct about how much weight the metric can bear, and Kwa's January 2026 note is the most concentrated statement of the limits.

Precision is the first casualty. METR's own example, from Kwa's January 2026 note and therefore a pre-correction TH1.0 figure, is Claude Opus 4.5 at a 50% horizon of around 4 hours 49 minutes with a 95% interval of 1 hour 49 minutes to 20 hours 25 minutes, about which Kwa writes: "I really have no idea whether Claude's true time horizon is 3.5h or 6.5h." Historically the error bars have been a factor of roughly 2 in each direction and are worse as the benchmark saturates, so a model being 10% or 20% above the previous best is not a meaningful distinction. [7]

The long-range extrapolation rests on a handful of frontier data points. The paper fit an exponential to eleven frontier models with an R-squared of at least 0.96, and noted that with so few points, fits with more parameters would be more likely to overfit than to help. That is a thin basis for projecting half a decade out, and small changes in the trend move the multi-year forecasts by years. [3]

The tasks are not representative of real work. They are well-specified, automatically scorable and self-contained, unlike the open-ended, underspecified, context-heavy jobs people actually do. METR studied this directly by scoring task "messiness" on 16 factors and found that a one-point increase reduced mean success rates by about 8.1%. The mean messiness score across HCAST and RE-Bench tasks is 3.2 out of 16 and none exceeds 8, where METR estimates that "write a good research paper" would score between 9 and 15. Trends over time were similar for the messy and clean subsets, so messiness appears to shift the level rather than the slope. [3]

Reliability is a separate axis from the headline number. A 50% horizon means even odds, and for anything you would actually delegate, the 80% horizon is the more honest figure. Kwa goes further: some reliability-critical and poorly verifiable tasks need 98% or better to be worth automating, and doubling the horizon does not double the degree of automation, because a more capable agent fails in more complex ways that cost more human labor per intervention. [7]

The scope is narrow. The original measurement is dominated by software and machine-learning engineering, and METR's own cross-domain follow-up found horizons 40-100x lower for visual computer use. Kwa's illustration of the point is that Claude Sonnet 4.5's real-world coffee-making time horizon is about two minutes. [7][9]

Baselining conventions could move the result by more than 1.25x on their own. METR aggregated multiple successful baselines with a geometric mean, where an arithmetic mean would have raised task lengths by roughly 25%; it excluded failed baselines rather than treating an X-hour failure as evidence the task takes longer than X hours, which would have raised them further; and its baseliner pool, while skilled, was not made up of frontier lab engineers, who would be faster. [7]

Finally, the metric is only well defined for the kind of task it was built for. METR's FAQ notes that solving 1,000 separate one-hour math problems is not a 1,000-hour task but a one-hour task done 1,000 times, because it splits trivially across workers. The prototypical multi-hour task is one where each fix reveals a new problem that only makes sense given what was already tried. [4]

## Does the underlying score mean what it seems to?

Two METR studies attack the metric from below, asking whether an automatically scored success is the same thing as a useful piece of work.

In August 2025 David Rein compared algorithmic and holistic scoring on 18 real issues from the stdlib-js and hypothesis repositories, tasks that took human maintainers between 20 minutes and four hours (averaging 1.3 hours). An Inspect ReAct agent running Claude 3.7 Sonnet passed the maintainers' own test cases 38% of the time. On manual review of 15 of those pull requests, none were mergeable as-is. Every one had at least three of five defect categories (core functionality, test coverage, documentation, linting and formatting, general code quality), 60% had at least four, and 20% had all five. Rein estimated the PRs would take on average 42 minutes to fix, or 26 minutes for those that passed the tests, against the 1.3 hours the original maintainer spent. The framing is explicitly about reconciling the 1-hour time horizon with METR's finding that AI tools slowed experienced open-source developers down. [13]

In March 2026 a larger follow-up put 296 AI-generated pull requests in front of four active maintainers from scikit-learn, Sphinx and pytest, covering 95 of SWE-bench Verified's 500 issues. Roughly half of the test-passing patches would not have been merged. Maintainer merge decisions ran about 24 percentage points below the automated grader's scores, and improved about 9.6 percentage points per year more slowly, though METR calls the rate result suggestive rather than solid. The authors are careful about what this does and does not show: agents got one shot with no feedback loop, so this is not evidence of a fundamental capability ceiling, only that a naive reading of benchmark scores overstates real usefulness. [16]

## MirrorCode

MirrorCode, released as preliminary results in April 2026, is a benchmark that METR funded and co-developed with Epoch AI and that cuts the other way. Agents are asked to reimplement a command-line program exactly, with execute-only access to the original binary and no source code, graded against hundreds to thousands of end-to-end tests. Claude Opus 4.6 fully reimplemented gotree, a bioinformatics toolkit of roughly 16,000 lines of Go covering about 40 subcommands, working autonomously and without internet access. Epoch explored inference budgets of up to one billion tokens per task, which it prices at around $550. Four researchers who had worked on the benchmark estimated independently that a skilled engineer would need 1.5-2.5, 13-17, 3 and 13 weeks respectively for the same job. [18][19]

Epoch's own comparison to the time-horizon numbers is the honest one: these apparent horizons are far longer than METR's roughly 12-hour estimate for Claude Opus 4.6, and the authors attribute the gap to the specification setup. A pre-existing binary is an unusually precise, programmatically checkable specification, which is not how most software is written. Their conclusion is that "it may be difficult to distill AI's software engineering capabilities to a single time horizon". The results are also preliminary, with no human baseline yet for the long tasks and an acknowledged memorisation risk. [19]

## Related metrics

METR has begun publishing alternatives that address the two structural weaknesses of time horizon: binary scoring throws away information, and the metric does not specify a budget.

**Expenditure horizon**, introduced in July 2026 by Tom Cunningham, Manish Shetty, Vincent Cheng and Nate Rush, measures optimization ability rather than task length. It is the dollar figure at which an agent's returns to spending stop beating a human's: plot score against expenditure for both, and read off where the curves cross. Applied to the NanoGPT speedrun, METR estimated from interviews with prolific contributors that each further 1% improvement costs roughly 16 hours of labor, about $2,400 at $150 per hour. Six agentic optimization runs starting from record #78 (85.56 seconds of training time, March 2026) produced expenditure horizons of $0 to $3,000 after more than $10,000 of spending. GPT-5 and Claude Opus 4.1 showed apparent progress that vanished on revalidation; GPT-5.5 and Opus 4.8 showed real gains. METR's own reading is that autonomous optimization does not yet have dramatic effects on AI R&D progress for this problem. Expenditure horizon is not a time horizon and the two are not interchangeable. [20]

**Metrics of Agent Ability**, a July 2026 survey by Cunningham, sets out a taxonomy of ways to compare agents when performance depends on spending. It separates agent-only metrics (score at a fixed budget, score at the practical plateau, spend needed to hit a target score, returns to expenditure) from human-grounded ones, and places time horizon among the latter as a binary human-grounded metric: many per-task binary comparisons against a human threshold, combined into a single human-equivalent time. The survey's continuous time horizon variant, comparing the agent's actual score against the whole human curve rather than one threshold, is noted as far more statistically efficient. [21]

## Reception and criticism

The chart became a fixture of AI discourse well beyond its intended audience, which METR staff describe with visible ambivalence. MIT Technology Review devoted a February 2026 explainer to it under the title "This is the most misunderstood graph in AI", reporting that the December 2025 Claude Opus 4.5 result drew reactions from Anthropic staff ranging from a researcher redirecting his research programme to an employee posting "mom come pick me up i'm scared". [10]

Sydney Von Arx of METR's technical staff told the magazine that "there are a bunch of ways that people are reading too much into the graph", and Kwa was blunt about the prospects for correcting it: "I think the hype machine will basically, whatever we do, just strip out all the caveats." Von Arx told the magazine "You should absolutely not tie your life to this graph", adding "But also, I bet that this trend is gonna hold." [10]

Two external criticisms recur. Inioluwa Deborah Raji of UC Berkeley questions the premise that human time proxies difficulty: "I don't think it's necessarily a given fact that because something takes longer, it's going to be a harder task." Daniel Kang of the University of Illinois Urbana-Champaign makes the generalization objection, that a model getting better at coding does not automatically get better at anything else, while also calling the study "one of the most carefully designed studies in the literature for this kind of work". METR's cross-domain follow-up partly answers Kang, but as the article notes, that work was less formal than the original. [10]

The chart's largest public role has been in forecasting. [AI 2027](https://aiwiki.ai/wiki/ai_2027) cited it extensively; Von Arx's comment on this is that "it's a little weird when the way lots of people are familiar with your work is this pretty opinionated interpretation". Sequoia Capital used it in a post arguing that AI capable of acting as an employee or contractor is imminent. The original blog post also drew detailed outside analysis, including the LessWrong piece "Interpreting the METR Time Horizons Post". [8][10]

METR researchers have themselves pushed back on how much forecasting weight the metric can carry. In February 2026 Kwa published a deliberately simplified 8-parameter takeoff model as an alternative to the 33-parameter AI Futures model. Kwa's model gives a median prediction of more than 99% automation of AI R&D in late 2032, and most of its simulations produce a 1,000x to 10,000,000x increase in AI efficiency by 2035. His stated reason for building it is that the AI Futures model "is extremely sensitive to time horizon in a way I wouldn't endorse": its doubling difficulty growth factor, which governs how superexponential future horizon growth is, can move the date of the first automated coder between 2028 and 2049, and Kwa suspects time horizon is too poorly defined to pin that parameter down. [17]

## Relation to safety forecasting and METR

The time horizon has become one of the more cited inputs to AI capability forecasting because it converts model progress into a quantity that bears on autonomy. The longer the tasks a system can complete unsupervised, the more plausible it is that the system could carry out extended, multi-step plans on its own, which is part of why METR, an organization focused on evaluating frontier models for dangerous capabilities, designed the metric in the first place. [1][2]

METR's GPT-5 report shows how the metric is used operationally rather than rhetorically. METR set out rough thresholds of concern in horizon terms: an 80% time horizon above 8 hours on high-context software engineering tasks, or a 50% time horizon above 40 hours on software and machine-learning tasks, would be the point at which a system might be approaching the capability to meaningfully accelerate AI development or to sustain rogue replication. Against those thresholds, GPT-5's measured 2 hours 17 minutes was judged far short, even taking the upper confidence bound and adding a further doubling. METR is explicit that "the link between our broad time horizon estimates and the particular threat models is very uncertain" and that the thresholds will move as understanding improves. [5]

Responsible use of the metric, in METR's own framing, means carrying the caveats along rather than treating the doubling line as a law of nature. [7][10]

## References

1. Kwa, T., West, B., et al. "Measuring AI Ability to Complete Long Software Tasks." arXiv:2503.14499, submitted 18 March 2025, last revised 10 July 2026. https://arxiv.org/abs/2503.14499
2. METR. "Measuring AI Ability to Complete Long Tasks." METR blog, 19 March 2025. https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
3. Kwa, T., West, B., et al. "Measuring AI Ability to Complete Long Tasks" (full text, v1). arXiv HTML, 2025. https://arxiv.org/html/2503.14499v1
4. METR. "Task-Completion Time Horizons of Frontier AI Models" (live tracker and FAQ; last updated 8 May 2026). https://metr.org/time-horizons/
5. METR. "Details about METR's evaluation of OpenAI GPT-5." August 2025. https://metr.org/evaluations/gpt-5-report/
6. METR. "Time Horizon 1.1." METR blog, 29 January 2026. https://metr.org/blog/2026-1-29-time-horizon-1-1/
7. Kwa, T. "Clarifying limitations of time horizon." METR notes, 22 January 2026. https://metr.org/notes/2026-01-22-time-horizon-limitations/
8. "Interpreting the METR Time Horizons Post." LessWrong, 2025. https://www.lesswrong.com/posts/fRiqwFPiaasKxtJuZ/interpreting-the-metr-time-horizons-post
9. Kwa, T. and Cheng, V. "How Does Time Horizon Vary Across Domains?" METR blog, 14 July 2025. https://metr.org/blog/2025-07-14-how-does-time-horizon-vary-across-domains/
10. Huckins, G. "This is the most misunderstood graph in AI." MIT Technology Review, 5 February 2026. https://www.technologyreview.com/2026/02/05/1132254/this-is-the-most-misunderstood-graph-in-ai/
11. Epoch AI. "METR Time Horizons." https://epoch.ai/benchmarks/metr-time-horizons
12. METR: Measuring AI Ability to Complete Long Tasks (crosspost). AI Alignment Forum, 2025. https://www.alignmentforum.org/posts/deesrjitvXM4xYGZd/metr-measuring-ai-ability-to-complete-long-tasks
13. Rein, D. "Research Update: Algorithmic vs. Holistic Evaluation." METR blog, 13 August 2025. https://metr.org/blog/2025-08-12-research-update-towards-reconciling-slowdown-with-time-horizons/
14. Jurkovic, N. "Measuring Time Horizon using Claude Code and Codex." METR notes, 13 February 2026. https://metr.org/notes/2026-02-13-measuring-time-horizon-using-claude-code-and-codex/
15. Barry, A. "Impact of modelling assumptions on time horizon results." METR notes, 20 March 2026. https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/
16. Whitfill, P., Wu, C., Becker, J. and Rush, N. "Many SWE-bench-Passing PRs Would Not Be Merged into Main." METR notes, 10 March 2026. https://metr.org/notes/2026-03-10-many-swe-bench-passing-prs-would-not-be-merged-into-main/
17. Kwa, T. "A simpler AI timelines model predicts 99% AI R&D automation in ~2032." METR notes, 10 February 2026. https://metr.org/notes/2026-02-10-simpler-ai-timelines-model/
18. Rein, D. "MirrorCode: Evidence that AI can already do some weeks-long coding tasks." METR blog, 10 April 2026. https://metr.org/blog/2026-04-10-mirrorcode-preliminary-results/
19. Adamczewski, T., et al. "MirrorCode: Evidence that AI can already do some weeks-long coding tasks." Epoch AI, April 2026. https://epoch.ai/publications/mirrorcode-preliminary-results/
20. Cunningham, T., Shetty, M., Cheng, V. and Rush, N. "Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT." METR blog, 21 July 2026. https://metr.org/blog/2026-07-21-expenditure-horizon/
21. Cunningham, T. "Metrics of Agent Ability." METR notes, 24 July 2026. https://metr.org/notes/2026-07-24-metrics-of-model-ability/
22. METR. "Inspect-Hawk" (source repository, MIT license). GitHub. https://github.com/METR/hawk
23. METR. Home page (Hawk description). https://metr.org/