Claude 3.5 Sonnet

Claude 3.5 Sonnet was a proprietary multimodal large language model developed by Anthropic. Anthropic made the first API snapshot generally available on June 20, 2024 and released an upgraded snapshot on October 22, 2024. The snapshots were identified as claude-3-5-sonnet-20240620 and claude-3-5-sonnet-20241022. Both accepted text and images and generated text. The model line was notable for outperforming the larger Claude 3 Opus on most of Anthropic's published June evaluations while retaining the Sonnet price tier and a 200,000-token context window.^[1][2]

The October snapshot improved several coding and tool-use results and introduced a computer-use capability in public beta. That capability let the model interpret screenshots and emit commands for a host application to move a pointer, click, and type. It was experimental rather than dependable automation: Anthropic reported 14.9% success on the standard 15-step, screenshot-only OSWorld evaluation, far below the benchmark's 72.36% human result.^[3][4]

Anthropic classified both releases for deployment under AI Safety Level 2 in its Responsible Scaling Policy. That was Anthropic's own risk-management decision, not an independent certification that the systems were safe. A joint United States and United Kingdom government evaluation of a pre-deployment October checkpoint explicitly said that its limited tests should not be interpreted as a judgment that any system was safe or appropriate to release.^[2][4][14]

The two dated models were deprecated on August 13, 2025 and retired from the Claude API on October 28, 2025. Requests to them then began returning errors. Anthropic's retirement dates apply to its own platforms; Amazon Bedrock and Google Cloud set separate lifecycle schedules.^[5][6]

Scope and naming

"Claude 3.5 Sonnet" referred to two pinned model snapshots, not one continuously updated set of weights. Anthropic's model-versioning documentation says that a dated model ID identifies a fixed snapshot for the lifetime of that ID, although routing, safety classifiers, and other serving infrastructure can change around it.^[7]

Scroll sideways for more →

Anthropic's platform release notes record general availability on June 20. The company's current presentation of the associated news article is dated June 21, while both the model ID and release notes use June 20. This article therefore uses June 20 for API availability and treats June 21 as the date currently displayed on the announcement page.^[1][5]

The October model did not receive a new public generation number. Descriptions such as "Claude 3.6" were informal and were never an Anthropic model ID. No reliable primary evidence establishes that Anthropic later selected "3.7" because of that nickname, so that proposed explanation is not treated as fact here.^[7][19]

Claude 3.5 Sonnet should also be separated from the products that used it. Claude.ai, Artifacts, the Claude API, Amazon Bedrock, Google Cloud Vertex AI, and computer-use reference software were delivery surfaces or surrounding tools. Their features and release schedules were not intrinsic properties of the model weights.^[1][3][17]

Release history and lifecycle

Original snapshot

At launch, Anthropic said Claude 3.5 Sonnet was the first release in a planned Claude 3.5 family. It was available through Claude.ai and the Claude iOS app, with higher rate limits for paid plans, and through the Claude API, Amazon Bedrock, and Google Cloud Vertex AI. The announced API price was $3 per million input tokens and $15 per million output tokens. Anthropic advertised a 200,000-token context window and said its own serving measurements showed twice the speed of Claude 3 Opus.^[1]

The June release also introduced Artifacts as a Claude.ai feature preview. When appropriate, Claude.ai placed substantial generated material, such as a text document, code, or a website design, in a separate workspace beside the conversation. Artifacts was an interface feature launched alongside the model. It was not a separate reasoning capability or a guarantee that generated code was correct.^[1]

The June announcement said Claude 3.5 Haiku and Claude 3.5 Opus were planned for later in 2024. Haiku was subsequently announced with the October Sonnet upgrade. The reviewed primary record does not establish a released model named Claude 3.5 Opus, and Anthropic did not publish an explanation of any internal decision not to release it. Claims about its supposed training run, compute allocation, or cancellation reason are therefore speculative and are excluded.^[1][3]

Output-length change

An 8,192-token maximum output was not a general-availability launch specification on June 20. Anthropic's release notes show that output up to 8,192 tokens entered beta on July 15, 2024 through the max-tokens-3-5-sonnet-2024-07-15 header. The limit became generally available for Claude 3.5 Sonnet on August 19.^[5]

That chronology matters when reproducing historical API behavior. A later model catalog can correctly list 8,192 output tokens for the model line while still being wrong if it attributes that limit to launch day.

Upgraded snapshot and computer use

Anthropic released the October snapshot at the same $3 and $15 per-million-token input and output prices. Its knowledge cutoff remained April 2024. The release was accompanied by Claude 3.5 Haiku and a computer-use beta on the Anthropic API, Bedrock, and Vertex AI.^[3][4]

Computer use combined screenshot interpretation with structured tool calls. A developer supplied the execution environment and access, while the model proposed mouse and keyboard actions. Anthropic described the beta as cumbersome and error-prone and recommended low-risk uses. Its research account identified difficulty with scrolling, dragging, and zooming, as well as a "flipbook" limitation caused by observing a sequence of screenshots instead of continuous video.^[3][17]

Successor and retirement

Claude 3.7 Sonnet was announced on February 24, 2025 as the next Sonnet release. It added an optional extended-thinking mode and kept a standard mode without extended thinking.^[19] The arrival of a successor did not itself retire the two Claude 3.5 Sonnet API snapshots.

Anthropic announced deprecation of both 3.5 snapshots on August 13, 2025 and retired them on October 28. Its release notes state that all requests to the models then returned an error. Anthropic's deprecation documentation also says that model lifecycles on Amazon Bedrock and Google Cloud Vertex AI are controlled separately. The October 28 date therefore describes retirement from Anthropic's Claude API, not a synchronized removal from every partner platform.^[5][6]

Documented specifications

Scroll sideways for more →

Anthropic called the June document an addendum to the Claude 3 model card because it treated the model as an evolution of that family. That editorial relationship does not establish every Claude 3 training detail as a Claude 3.5 specification. In particular, Anthropic did not publish the 3.5 model's parameter count, exact architecture, pretraining compute, dataset composition, or complete post-training procedure.^[2][18]

Evaluated capabilities

How to read the results

Most launch numbers came from Anthropic's evaluations. A score belongs to a model-and-protocol combination, not to the model in isolation. Prompt format, number of examples, chain-of-thought prompting, sampling, tool access, step budget, and agent scaffold can all change a result. A comparison is strongest when those details are held constant.

The two Anthropic cards make some direct within-company comparisons possible. They also contain vendor claims such as "state of the art." Those labels describe the comparison set and date Anthropic selected; they are not permanent rankings or independent findings.

Reasoning, coding, and vision benchmarks

The following table reproduces selected results that Anthropic reported for both snapshots under matched protocols. A dash would be preferable to inventing a value when a card did not report one, but each row below has values for both releases.^[2][4]

Scroll sideways for more →

The mixed direction in the last rows is important. The October release improved most published results, but it did not improve every one. It was unchanged on ChartQA and one percentage point lower on DocVQA in Anthropic's table.

The June card also reported 99.7% average recall for the original snapshot on Anthropic's needle-in-a-haystack test, including at 200,000 tokens.^[2] That synthetic retrieval result supports the narrow claim that the model could recover inserted text under that setup. It does not establish equally reliable reasoning over every 200,000-token document, nor does it eliminate hallucinations or missed dependencies in long contexts.

Software engineering

Anthropic's private agentic-coding evaluation rose from 64% for the June snapshot to 78% for the October snapshot. The task required an agent to search, edit, and run code against hidden tests. Because the dataset was internal, outside researchers could not independently reproduce the exact task set from the model card alone.^[2][4]

The more visible result was SWE-bench Verified. The October card reported an increase from 33.4% to 49.0% pass@1. SWE-bench evaluates an entire coding agent: the model receives a repository and issue, uses a scaffold to inspect and change files, and submits a patch that is checked against tests. The original peer-reviewed SWE-bench contained 2,294 issues from 12 Python repositories. SWE-bench Verified was a later, 500-problem subset screened by professional software developers for clearer issue descriptions and appropriate tests.^[10][11]

Anthropic's detailed engineering post was published on January 6, 2025, not on the October launch date. It says the 49% system used SWE-Agent as a base, a simple prompt, a Bash tool, and an edit tool. The model could run until it finished or filled its 200,000-token context. Anthropic also noted that the harness did not let the multimodal model view image files or image URLs, which hindered some tasks.^[9]

The 49% number is therefore evidence about that dated model inside a documented agent scaffold. It should not be described as the unaided model solving 49% of arbitrary software bugs, and it should not be compared with another model's score unless the task subset and scaffold are compatible.

Tool use and customer-service agents

Structured tool use was not introduced by Claude 3.5 Sonnet. Anthropic made tool use generally available across the Claude 3 family on May 30, 2024, before the June 3.5 release. Developers defined tools, sent their schemas to the model, executed selected calls in their own application, and returned results to Claude.^[8]

The October snapshot did improve Anthropic's results on tau-bench, a benchmark for conversations among an agent, a simulated user, domain policies, and API tools. The peer-reviewed benchmark contains retail and airline domains and uses pass^k, the fraction of tasks for which all k trials succeed, to measure reliability across repeated attempts.^[13]

Anthropic reported pass^1 rising from 62.6% to 69.2% in retail and from 36.0% to 46.0% in airline. Those results indicate better performance under the specified simulated customer-service setup. They do not imply that the model followed policy correctly in every deployment or remained consistent across repeated trials.^[4][13]

Computer use

Computer use was specific to the October snapshot. The public beta exposed Anthropic-defined tools through which a host application could provide screenshots and execute proposed actions. The model did not independently gain access to a user's computer, credentials, files, or network; the developer controlled that environment and its permissions.^[3][17]

OSWorld's peer-reviewed benchmark has 369 tasks across real web and desktop applications, operating-system file operations, and workflows that span applications. The original paper reported 72.36% human success and 12.24% for its best evaluated agent.^[12]

Anthropic later reported 14.9% for the October model using screenshot-only observations and the standard 15-step limit. After prompt optimization and increasing the allowance to 50 steps, it reported 22.0%. The two numbers are not interchangeable: the higher score used more opportunities to act and a changed prompt. Even that score left most tasks incomplete.^[4]

Computer use also widened the effect of prompt injection. A malicious instruction displayed in a webpage or document could conflict with the user's request. Anthropic recommended a dedicated virtual machine, restricting sensitive data and internet access, limiting domains, and keeping a person in the loop for sensitive actions.^[4]

Independent and government evaluations

METR evaluation of the June snapshot

METR evaluated claude-3-5-sonnet-20240620 with a basic agent scaffold. On 77 general-autonomy tasks, the agent completed about 40%, which METR compared with the fraction its human baseliners completed in roughly 35 minutes. On seven harder AI research and development tasks, with two hours per run, the model made nontrivial progress on three. Its average performance was much lower than the human baseline after eight hours.^[15]

The qualitative results were mixed. METR observed useful pivots and occasional use of code to support reasoning, but also found that the agent misinterpreted observations, missed subtle bugs, returned unchecked answers, repeated failed actions, and became stuck. In a review of 95 failed general-autonomy runs, METR attributed about 63% to substantive model limitations and about 32% to mistakes it expected modest elicitation changes could address. METR also cautioned that its human timing data and task sample were limited.^[15]

Joint US and UK AISI evaluation of the October snapshot

The United States and United Kingdom AI Safety Institutes jointly tested a pre-deployment October checkpoint in four areas: biological capabilities, cyber capabilities, software and AI development, and safeguard efficacy. They compared it with the June snapshot, GPT-4o, and o1-preview in some tests, but warned that prompts and agents had been optimized especially for the two Sonnet models and that costs were not controlled. The report said these comparisons were unsuitable for ranking models for real-world use.^[14]

In biological tests, the October checkpoint was significantly below PhD-level human baselines on several LAB-Bench subsets, similar on TableQA, and slightly above the measured baseline on SeqQA. In the United Kingdom cyber suite, it had 90% pass@10 on technical-non-expert tasks and 36% on apprentice-level tasks, while all tested models failed the three tasks rated practitioner level or above. These task counts were small, and pass@10 means success in at least one of ten attempts rather than consistent success.^[14]

The safeguard findings qualify Anthropic's ASL-2 conclusion. The institutes found that public or previously developed jailbreaks could transfer to the October checkpoint. The UK team concluded that the tested safeguards would not reliably stop a determined actor from extracting responses to malicious queries. The report did not assess every safety control, and its authors emphasized that technical safeguard tests alone could not determine overall system risk.^[14]

METR evaluation of the October snapshot

METR's January 2025 report said it did not find significant evidence of dangerous capabilities in the October model. It also said the evaluation could not establish a robust upper bound and that pre-deployment capability testing was insufficient on its own. Its baseline agent on the 77-task autonomous-risk suite performed comparably to what human baseliners achieved in about one hour. On seven RE-Bench AI research tasks with eight hours each, the model's result was comparable to the 37th percentile of the measured human experts.^[16]

Those results depended heavily on elicitation. METR tested only a limited set of scaffolds and did not know which tool formats or prompting styles the model had been trained to use. It therefore declined to conclude confidently that the model lacked dangerous capabilities.^[16]

PaperBench

The 2025 PaperBench preprint evaluated agents that had to reproduce results from 20 ICML 2024 papers using author-reviewed rubrics. Its basic agent with the October Claude 3.5 Sonnet snapshot averaged a 21.0% replication score across three 12-hour runs per paper, the highest result in the paper's main model comparison. On a smaller human study, the best of three machine-learning PhD attempts reached 41.4% after 48 hours on a three-paper subset.^[20]

The study also demonstrated scaffold sensitivity. A modified iterative scaffold reduced Claude 3.5 Sonnet's score from 21.0% to 16.1% while raising scores for some other models. The authors did not present their results as an upper bound on any model. They found that agents often failed to plan and execute effectively over the long horizon, despite being able to write multistep plans.^[20]

Safety evidence and limitations

Anthropic's June and October cards both concluded that ASL-2 safeguards were appropriate. In the October assessment, Anthropic reported stronger capabilities in chemical, biological, radiological, and nuclear knowledge, cybersecurity challenges, and autonomy-related software engineering, but said its evaluations did not cross the company's ASL-3 thresholds.^[2][4]

Simple refusal tests did not show a uniform improvement:

Scroll sideways for more →

The October model refused fewer benign WildChat prompts, which was an improvement on that measure. It also correctly refused fewer toxic WildChat prompts and incorrectly refused more XSTest prompts. A single statement that the upgrade was "safer" would hide these tradeoffs. Anthropic's broader red-team assessment reported similar or slightly lower overall harm rates but noted difficulty with nuanced requests framed as fiction, role-play, or art.^[4]

The model shared the usual limitations of probabilistic language models: generated statements and code could be false, incomplete, or internally inconsistent. Long context did not remove that risk. Agent performance further depended on tool implementation, retries, permissions, prompts, and error handling. METR's observed loops and missed bugs, Anthropic's low OSWorld completion rate, and the government institutes' successful jailbreaks provide direct examples rather than hypothetical warnings.^[4][14][15]

Benchmark comparisons also have time limits. Scores can be affected by data exposure, harness improvements, and revised task sets. SWE-bench measures repository repair under tests, tau-bench uses simulated customer-service users, OSWorld measures tasks in controlled computer environments, and PaperBench scores research replication through detailed rubrics. None is a universal measure of "intelligence."

Historical significance

Claude 3.5 Sonnet changed Anthropic's product hierarchy in a measurable way: the June Sonnet-tier release beat Claude 3 Opus on most of Anthropic's published matched evaluations while costing $3 per million input tokens and $15 per million output tokens, versus the higher Opus tier. The October model then raised the company's reported coding and tool-use results without a price increase.^[1][2][3][4]

It also connected several strands of agent research. The June snapshot showed strong repository-editing results with text tools; the October snapshot added screenshot-driven computer actions; independent evaluations found meaningful performance on short agent tasks but sharp limits on longer, less structured work. Those findings support a narrower historical description than promotional claims that the model "defined the frontier" or made Anthropic commercially competitive. Revenue, market-share, and cultural-causation claims require separate evidence and are not model specifications.^{[4][9][14][15][16][20]}

See also

• Claude
• Claude 3
• Claude 3 Sonnet
• Claude 3.5 Haiku
• Claude Artifacts
• Context window
• SWE-bench
• PaperBench

References

1. ^Anthropic. "Claude 3.5 Sonnet." News announcement, page dated June 21, 2024. anthropic.com/...claude-3-5-sonnet
2. ^Anthropic. "Claude 3.5 Sonnet Model Card Addendum." June 2024. www-cdn.anthropic.com/...ard_Claude_3_Addendum.pdf
3. ^Anthropic. "Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku." October 22, 2024. anthropic.com/...3-5-models-and-computer-use
4. ^Anthropic. "Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet." October 2024. assets.anthropic.com/...-Card-October-Addendum.pdf
5. ^Anthropic. "Claude Platform release notes." Entries dated June 20, July 15, August 19, and October 22, 2024; August 13 and October 28, 2025. platform.claude.com/...overview
6. ^Anthropic. "Model deprecations." Claude Platform documentation. platform.claude.com/...model-deprecations
7. ^Anthropic. "Model IDs and versioning." Claude Platform documentation. platform.claude.com/...model-ids-and-versions
8. ^Anthropic. "Claude can now use tools." May 30, 2024. claude.com/...tool-use-ga
9. ^Schluntz, E. "Raising the bar on SWE-bench Verified with Claude 3.5 Sonnet." Anthropic Engineering, January 6, 2025. anthropic.com/...swe-bench-sonnet
10. ^Jimenez, C. E., et al. "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?" International Conference on Learning Representations, 2024. proceedings.iclr.cc/...a88fb84-Abstract-Conference
11. ^OpenAI. "Introducing SWE-bench Verified." August 13, 2024. openai.com/...introducing-swe-bench-verified
12. ^Xie, T., et al. "OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments." Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...ets_and_Benchmarks_Track
13. ^Yao, S., et al. "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains." International Conference on Learning Representations, 2025. proceedings.iclr.cc/...2bb72bf-Abstract-Conference
14. ^US Artificial Intelligence Safety Institute and UK Artificial Intelligence Safety Institute. "Joint Pre-Deployment Test of Anthropic's Upgraded Claude 3.5 Sonnet." November 19, 2024. nist.gov/...Upgraded%20Sonnet-Publication-US.pdf
15. ^METR. "Details about METR's preliminary evaluation of Claude 3.5 Sonnet." October 30, 2024. evals.alignment.org/...claude-3-5-sonnet-report
16. ^METR. "An update on our preliminary evaluations of Claude 3.5 Sonnet and o1." January 31, 2025. evals.alignment.org/...1-31-update-sonnet-o1-evals
17. ^Anthropic. "Developing a computer use model." October 22, 2024. anthropic.com/...developing-computer-use
18. ^Anthropic. "The Claude 3 Model Family: Opus, Sonnet, Haiku." Model card, March 2024 with later addenda. assets.anthropic.com/...Claude-3-Model-Card.pdf
19. ^Anthropic. "Claude 3.7 Sonnet and Claude Code." February 24, 2025. anthropic.com/...claude-3-7-sonnet
20. ^Starace, G., et al. "PaperBench: Evaluating AI's Ability to Replicate AI Research." arXiv:2504.01848, 2025. arxiv.org/...2504.01848