AI Agents

An AI agent is a software system that selects and performs actions in an environment in pursuit of an objective. In contemporary usage, the term often refers to a system built around a Large Language Model, but the model is only one component. The complete system also includes an observation interface, instructions and policies, an action interface, state, control logic, and rules governing when execution must stop or return to a person. A model that produces an answer is not automatically an agent; agency arises when outputs are interpreted as decisions or actions within an execution loop.

The term has a broader history in Artificial Intelligence. Classical accounts characterized intelligent agents through properties such as autonomy, reactivity, proactiveness, and social ability.^[1] Modern language-model agents instantiate only particular versions of those ideas. Their behavior depends on the model, prompts, tools, environment, stored state, and surrounding software. A result obtained by one scaffold therefore does not establish a general capability of the underlying model or of agents as a class.

AI agents range from tightly constrained assistants that ask before every consequential action to systems allowed to complete a bounded task without intermediate approval. Autonomy is consequently a property of a deployment configuration, not a binary label or evidence of general intelligence. Reliable evaluation must identify the task boundary, available actions, permissions, stopping conditions, and intervention policy. This article focuses on contemporary model-based agent systems. The broader Agent abstraction, the Agentic AI paradigm, and product-specific agents are adjacent topics.

Between 2023 and 2026 the engineering practice around agents changed substantially. Early systems chained prompts through library code that parsed free text into actions. Later systems delegated much of that work to models trained to emit structured tool calls, and to standardized interfaces for tools, repository instructions, and agent-to-agent messaging, several of which now sit under neutral foundations rather than single vendors. Software engineering became the first area where agents were sold at scale. Measured performance in less controlled settings remains well below both demonstration results and vendor descriptions, and the security properties of tool-using agents remain unresolved. This article covers the architecture, the interfaces, the evaluation problems, and the published evidence about deployment, current as of August 2026.

Scope and terminology

An agent can be described as a policy implemented in software: it receives an observation, uses available state to choose an action, obtains a new observation or outcome, and repeats until it reaches a terminal condition. In a language-model agent, observations and actions are commonly represented as text or structured data. An observation might contain a user request, a web page, a tool result, an error, or a screenshot. An action might be a message, a function call, a file edit, a browser operation, or a command sent to a robot.

This formulation separates several concepts that are often conflated:

• A Chatbot may generate conversational responses without acting outside the conversation.
• An Agentic workflow is a repeatable process that may contain one or more agent loops as well as deterministic steps.
• An agent scaffold is the software that assembles context, invokes a model, parses outputs, dispatches actions, records results, and enforces limits.
• An AI browser agent, AI coding agent, or Computer-use agent is distinguished primarily by its observation and action interfaces, not by a universal level of intelligence.
• An embodied agent in Robotics must connect high-level decisions to perception, control, and the physical constraints of a particular machine.

A widely used working distinction separates workflows from agents by where control lives. In a December 2024 engineering note, Anthropic defined workflows as "systems where LLMs and tools are orchestrated through predefined code paths" and agents as "systems where LLMs dynamically direct their own processes and tool usage," and recommended "finding the simplest solution possible, and only increasing complexity when needed."^[30] The same note argued that agents suit "open-ended problems where it's difficult or impossible to predict the required number of steps," and that they carry "higher costs, and the potential for compounding errors."^[30] This is a vendor's engineering guidance rather than a research finding, but the boundary it draws is useful: a system with a fixed number of model calls arranged by a programmer is easier to test, price, and audit than one whose control flow is decided at runtime by a model.

The word harness has become common for the scaffold surrounding a model, particularly in software-engineering contexts, where it names the command set, file access, editing tools, and feedback that a model is given. A Harness is a distinct artifact from the model and is versioned separately, which is why the same model reports different scores under different harnesses.

The word autonomous should state what human involvement has been removed. A system may choose tool calls by itself while still requiring approval for payments, external communication, deletion, or physical motion. It may also operate independently only inside a fixed test environment. Descriptions such as "fully autonomous" are incomplete unless they specify the action space, time horizon, permissions, supervision, and failure-handling process. Market research firms have described the practice of relabeling existing assistants, chatbots, and robotic process automation as agentic, with Gartner calling it "agent washing" and estimating in June 2025 that only about 130 of the thousands of vendors it reviewed offered products it considered genuinely agentic.^[56]

Historical development

Precursors and early language-model agents

Research on software agents predates large language models. Wooldridge and Jennings surveyed agent theories and architectures in 1995 and distinguished weak notions of agency, including reactive and goal-directed behavior, from stronger claims about knowledge, intention, or emotion.^[1] That distinction remains useful because an engineered system can behave purposefully without possessing the mental states that ordinary language may suggest.

Language models made natural-language instructions and observations a practical control interface. WebGPT placed a language model in a text-based browser, trained it to search and navigate, and required it to collect references for long-form question answering.^[2] The experiment was task-specific: its browsing environment, demonstrations, reward model, and evaluation protocol defined what the system could do.

Work on embodied control exposed the need to ground proposed actions in an environment. SayCan combined language-model scores for high-level instructions with value functions for available robot skills, so an action had to be both linguistically relevant and feasible for the tested robot.^[3] ReAct interleaved generated reasoning traces with task-specific actions in question answering and interactive environments, allowing new observations to affect subsequent decisions.^[4] Toolformer studied a different route, training a model to decide when and how to call a fixed set of APIs and how to incorporate their returned values.^[5]

These systems established reusable design patterns rather than a single agent architecture. Later work added explicit search, stored memories, execution feedback, role-based coordination, and richer computer interfaces. The resulting systems remain composites: their behavior can change when the same model is placed behind different tools, prompts, parsers, or permission rules.

Prompt-chaining frameworks, 2022 to 2023

The first widely used agent software was library code that arranged model calls into sequences and parsed free text into actions. LangChain, released in late 2022, popularized chains, tools, and text-parsing agent executors. Auto-GPT and BabyAGI, both released in 2023, made the recursive goal-decomposition loop legible to a general audience by letting a model write its own task list and work through it. Those projects attracted very large amounts of attention relative to their measured reliability, and the failure modes they exposed, including loops, budget exhaustion, and confidently wrong intermediate results, shaped later designs.

Two structural weaknesses defined this period. First, action selection depended on the model producing text that a regular expression or a parser could interpret, so a formatting error became a control-flow error. Second, the model had no training signal for the specific tool schemas it was shown, so tool choice quality varied with prompt wording. Research systems of the same period, including ChatDev, MetaGPT, and Reflexion, explored role specialization and stored feedback on top of the same fragile substrate.^[11][15]

Model-native tool use, 2024 to 2025

The decisive change was moving action selection from text parsing into the model interface itself. Function-calling APIs, introduced across major providers from 2023 onward, let a caller declare tool schemas and receive structured invocations rather than prose, which removed a large class of parsing failures. Structured outputs extended the same idea to arbitrary response shapes. Providers then added server-side execution loops, so that a single API call could run several tool round trips before returning; the OpenAI Responses API, released in March 2025, is one example of that pattern.

Training changed alongside the interfaces. Post-training for tool use, long context handling, and multi-step execution made models better at deciding when to call a tool, when to stop, and how to recover from a tool error. The practical consequence is that a large part of what earlier frameworks implemented in Python moved inside the model, and framework code shifted toward context management, permissions, persistence, and observability. Practitioners began describing this remaining work as Context engineering: deciding what enters the Context window at each step, what is summarized, what is offloaded to files, and what is discarded.

Agentic coding as the first commercial category

Software engineering became the first area where agents were sold at scale, for reasons that are structural rather than incidental. Code has cheap, fast, automatic verification through compilers, type checkers, linters, and test suites, so an agent can obtain a real reward signal without a human in the loop. Repositories provide a bounded environment. Errors are usually reversible through version control. The users are technical enough to supervise and to recover from failures.

Product forms converged on the terminal and the editor. SWE-agent established in a research setting that the specific commands and feedback exposed by an agent-computer interface materially changed measured performance.^[9] Commercial harnesses followed, including Claude Code, OpenAI Codex, Gemini CLI, Devin, and Cursor, alongside code-review and issue-resolution agents built into hosting platforms.

Independent revenue measurement for this category does not exist; the available figures are company statements. On its FY26 second-quarter earnings call on 28 January 2026, Microsoft chief executive Satya Nadella said that "all up now we have over 4.7 million paid Copilot subscribers, up 75% year-over-year" for GitHub Copilot.^[60] Such statements establish that products are being purchased. They do not establish that the agents deliver the productivity effects buyers expect, a question addressed separately below.

Computer use and browser agents

A parallel line of work gave agents the same interfaces people use. Anthropic released a computer use capability in public beta in October 2024, exposing screen, keyboard, and mouse control together with text-editor and shell tools (Anthropic Computer Use). OpenAI Operator followed in January 2025 as a research preview that drove a hosted browser, and was later folded into ChatGPT Agent. Google's Project Mariner and subsequent computer-use models pursued browser control with page-structure awareness rather than pixels alone. Open implementations such as Browser Use and UI-TARS made the same interfaces available outside vendor products.

Two observation strategies compete. Pixel-based control works anywhere a screen exists but demands a vision model and is sensitive to layout changes. Structure-based control reads the accessibility tree or the document object model, which is cheaper and more stable but only available where such a tree exists. Playwright MCP, a Microsoft-maintained server, takes the structural route by feeding agents the accessibility tree rather than screenshots, removing the need for a vision model in browser tasks.

Measured performance in this area has consistently trailed demonstrations. In Windows Agent Arena, the baseline agent Navi completed 19.5% of tasks against 74.5% for an unassisted human on the same task set.^[42] Web-agent results have shown a similar pattern when evaluation moved from curated offline suites to live websites.^[44]

Long-horizon execution and standardized interfaces

From 2025 the frontier of the field shifted from whether an agent could complete a step to how long it could keep working. METR proposed measuring the length of task, in human expert time, that a model completes with 50% reliability, and reported in March 2025 that this quantity had roughly doubled every seven months over the preceding six years.^[52] Its January 2026 revision, built on a larger task suite of 228 tasks and new evaluation infrastructure, reported the same 196-day doubling time for the full historical trend but faster recent progress: 131 days for models released since 2023 and 88.6 days for those since 2024, with a 50% time horizon of 320 minutes for the highest-scoring model measured, and wide confidence intervals throughout.^[53] METR itself cautions that "the trend in time horizon is somewhat sensitive to task composition."^[53]

Longer horizons made interfaces matter more, because an agent working for hours touches many tools, repositories, and services. The period from late 2024 through 2026 accordingly produced a set of standardized interfaces, several of which moved from single-vendor control into neutral foundations, discussed in the next section.

Architecture and execution

A practical agent normally contains at least six functions:

1. Task interpretation. The system converts a request into an objective, constraints, and a completion condition. Ambiguity at this stage can propagate through every later action.
2. Observation. Adapters turn environment state into model-readable input. They may expose selected text, database records, visual representations, or tool errors rather than the entire environment.
3. Decision. A model or other controller selects the next action. The controller may request one model completion, compare several proposals, invoke a planner, or follow deterministic rules for part of the process.
4. Execution. A dispatcher validates and performs an action through an operating-system command, browser control, code runtime, robotic skill, or Function calling interface.
5. State update. The system records outcomes, intermediate artifacts, remaining work, budgets, and errors.
6. Termination and escalation. The system decides whether the objective is met, whether another attempt is justified, or whether control should pass to a human.

The loop is closed when an executed action changes what the agent observes next. A plan written once and never revised is open-loop execution. Closed-loop execution can respond to errors and environmental changes, but feedback alone does not guarantee correction. If an observation is incomplete, misleading, or parsed incorrectly, another iteration may compound rather than repair the error.

The model-scaffold boundary is central to interpretation. The model proposes tokens. The scaffold decides which text becomes an executable action, what data enters the context window, which tools are visible, and what happens after a timeout or malformed result. API-Bank separated tool behavior into planning, tool retrieval, and API calling, illustrating that failure can occur at several interfaces rather than in one undifferentiated "reasoning" step.^[6] SWE-agent showed within software engineering that the commands and feedback exposed by an agent-computer interface materially affected the behavior and performance of the evaluated system.^[9]

An action interface should define types, required arguments, permissible values, error semantics, and side effects. Read operations, reversible writes, external communications, and destructive operations should not be treated as equivalent. Operations that may be retried need idempotency or deduplication controls so that an uncertain response does not produce a repeated payment, message, or deletion. A tool result should also indicate whether it reflects confirmed state, a partial result, or an estimate.

Context management as a first-class component

As horizons lengthened, the scarce resource stopped being model capability on a single step and became the context window. Long trajectories accumulate tool output, file contents, and prior reasoning faster than any window can hold, and naive truncation silently removes the constraints that governed earlier decisions. Four techniques are now common, and each has a distinct failure mode:

Scroll sideways for more →

Packaged implementations of this pattern now exist as libraries. Deep Agents, from LangChain, bundles a planning tool, ephemeral subagents, a virtual file system, and context-compression middleware into a single harness, generalizing an architecture first popularized by coding tools. Agent Skills address the same pressure from the opposite direction, storing procedural instructions on disk and loading them only when relevant, so that capability breadth does not cost context on every turn.

Planning and control

Agent planning spans several mechanisms. A reactive controller selects one action from the current observation. A plan-and-execute controller first proposes a sequence and then carries it out, possibly with replanning after each result. Search-based controllers generate alternatives, estimate their value, and retain or discard branches. Hierarchical systems split an objective into subtasks and may assign different controllers or tools to each level. Agent planning covers these mechanisms in more detail.

Chain-of-Thought prompting can expose intermediate text that a controller uses as a scratchpad, while ReAct alternates such text with environmental actions.^[4] Tree of Thoughts explored search over multiple intermediate text states and reported gains on three selected reasoning tasks.^[7] Those results do not establish a domain-independent planning algorithm: branching, state evaluation, and stopping rules were designed for the studied tasks.

Executable plans require more than plausible descriptions. In classical planning domains studied by Valmeekam and colleagues, evaluated language models frequently failed to produce valid plans autonomously. Model outputs were more useful as heuristic guidance when combined with sound planners and external verification.^[8] SayCan similarly constrained high-level proposals with learned feasibility estimates for a defined robot skill set.^[3] These findings support a general engineering distinction between proposing a possible next step and proving that the step is valid in the current state.

Verification can occur before or after execution. Preconditions can reject actions whose required state is absent. Static analyzers, type checkers, policy engines, or simulation can test a proposal before it changes the environment. Postconditions can inspect whether the intended state was actually reached. For long tasks, checkpoints make it possible to resume, roll back, or request review without replaying the entire trajectory.

More planning is not always better. Additional branches consume time and model calls, and a weak evaluator may rank an incorrect branch above a correct one. Repeated reflection can also change correct answers. In experiments focused on intrinsic self-correction of reasoning without external feedback, Huang and colleagues found that prompting evaluated models to revise themselves did not reliably improve the studied tasks and sometimes reduced performance.^[12] A retry policy should therefore be tied to new evidence, an independent check, or a defined failure signal rather than an assumption that another generation is inherently corrective.

Long-horizon control adds a budget problem that short tasks do not have. Once an agent may run for hours, the operator must decide in advance how much time, money, and tool access a single objective is worth, and what happens when a budget is exhausted mid-task. Durable execution runtimes address the mechanical half of this by persisting agent state so that a crashed or paused run resumes rather than restarts; LangGraph is one implementation, with checkpointing and interrupt points built into the runtime rather than added by application code.^[38] The policy half, deciding whether an unfinished trajectory should be resumed, abandoned, or escalated, remains a human decision in most deployments.

Tool use and interfaces

Tool use extends the action space beyond text generation. Tools can provide retrieval, calculation, code execution, database access, messaging, or control of external devices. Toolformer demonstrated learned selection and invocation for a fixed set of APIs,^[5] while API-Bank evaluated planning, retrieval, and calling over runnable tools.^[6] These studies support more limited claims than "models know how to use any tool": success depends on tool descriptions, examples, argument schemas, returned observations, and the distribution of tasks.

A tool layer usually performs four distinct operations:

• discovery or selection of a tool;
• construction and validation of arguments;
• execution with an authenticated identity and bounded permissions;
• interpretation of the result and its effect on the task state.

Errors at these stages have different remedies. Better retrieval may help when the correct tool is not selected, but it will not fix an unauthorized permission scope. Schema validation may catch a malformed argument but not a valid request based on a false premise. A successful HTTP response does not prove that the user's objective was satisfied.

External knowledge can be added with Retrieval-Augmented Generation, and embeddings may be stored in a Vector database. Retrieval is not itself an action policy. The controller must still decide when to retrieve, how to rank evidence, how to handle conflicting or stale records, and whether a retrieved instruction is data or authority.

Tool count is itself a design variable. Every tool declaration consumes context and adds a candidate the model can select incorrectly, so large tool catalogs are usually filtered by task, exposed through a search tool, or grouped behind a smaller number of higher-level operations. This is one reason protocol-level tool discovery matters: it moves the catalog out of the prompt and into a queryable service.

Protocols and interoperability

Between 2024 and 2026 the industry converged on a small number of open interfaces, and most of them changed hands. Because governance determines who can make breaking changes, it is worth stating separately from technical design. The table records the position as of August 2026.

Scroll sideways for more →

Model Context Protocol

MCP standardizes how an application supplies a model with tools, data, and reusable prompts. A host application runs one or more clients, each connected to a server that exposes capabilities; the same server therefore works with any compliant client. This is the layer at which most agent tooling is now integrated, and by December 2025 the Linux Foundation cited "more than 10,000 published MCP servers."^[32] An MCP server can wrap an internal database, a SaaS API, a browser, or a local file system.

Anthropic donated MCP to the newly formed Agentic AI Foundation on 9 December 2025. The foundation is a directed fund under the Linux Foundation, co-founded by Anthropic, Block, and OpenAI, with Amazon Web Services, Bloomberg, Cloudflare, Google, and Microsoft among its platinum members.^[32] The MCP maintainers stated that the transfer did not change technical control: "The governance model we introduced earlier this year continues as is," with decisions resting with the existing maintainers and community input arriving through the project's enhancement-proposal process.^[31] The two other anchor projects were goose, an agent framework contributed by Block, and AGENTS.md, contributed by OpenAI.^[32]

MCP revisions are date-stamped strings of the form YYYY-MM-DD, marking the last date on which backward-incompatible changes were made; the version is not incremented for compatible improvements.^[34] The current revision as of August 2026 is 2026-07-28, and it is the largest break in the protocol's history. It converts MCP "from a bidirectional stateful protocol into a request/response stateless protocol," retires the initialize handshake and the session header, moves method and tool names into HTTP headers so gateways can route without parsing bodies, adds cacheable list results, and formalizes an extensions framework covering server-rendered interfaces and long-running tasks.^[33] Authorization moved toward client metadata documents and away from dynamic client registration, with issuer validation per RFC 9207.^[33] The revision also introduced a formal feature lifecycle: deprecated features remain in the specification for at least twelve months before they become eligible for removal.^[33][34] Revisions through 2025-11-25 used the handshake-based design, and the specification documents backward compatibility with them.^[34]

The practical significance of the stateless rewrite is operational rather than conceptual. A stateful protocol requires sticky routing and per-session memory in every intermediary; a stateless one can be served by ordinary load balancers and caches. That matters once an organization runs thousands of servers rather than a handful on a developer laptop.

Agent-to-agent interfaces

A2A addresses a different layer: how independently built and independently operated agents advertise capabilities and exchange tasks. Google announced it in April 2025 and donated it to the Linux Foundation on 23 June 2025 at the Open Source Summit North America, with Amazon Web Services, Cisco, Microsoft, Salesforce, SAP, and ServiceNow among the founding participants.^[35] Google reported version 1.0, described as the first stable production-ready release, in March 2026, and more than 100 supporting companies by the protocol's first anniversary in April 2026.^[36] Related specifications developed alongside it cover payments and user interfaces, including the Agent Payments Protocol.^[36] A separate industry effort, the Agentic Commerce Protocol, addresses purchasing flows from the merchant side.

The distinction between MCP and A2A is often blurred in marketing material. MCP is about giving one agent access to capabilities. A2A is about one agent delegating to another that it does not control, which raises trust, identity, billing, and liability questions that a tool call does not. Neither protocol makes a remote counterparty trustworthy, and both leave authorization to the deployment.

Repository and interface conventions

AGENTS.md is not a wire protocol but a file convention: a Markdown file at the root of a repository containing build commands, test instructions, code conventions, and constraints for coding agents. Its value comes entirely from convergence, since a project maintains one file instead of a separate configuration for each tool. The Linux Foundation reported adoption by "more than 60,000 open source projects" at the time of the Agentic AI Foundation announcement.^[32]

AG-UI standardizes the event stream between an agent backend and a front-end application, covering streaming text, tool-call status, shared state, and generative interface elements. It is MIT-licensed and maintained by its own project rather than a foundation, with integrations spanning several agent frameworks.^[37] NLWeb, a Microsoft open-source project announced at Build in May 2025, approaches interoperability from the publisher side, letting a website expose its own content as a natural-language endpoint that also functions as an MCP server.

None of these interfaces changes what an agent can do. They change how many bespoke integrations an organization has to write and maintain, and they concentrate risk: a flaw in a widely deployed server or convention reaches every agent that adopted it.

Frameworks and harnesses

Agent software divides into two categories that are frequently confused. A framework is a library used to build an agent, and its users are developers. A harness is a finished agent application whose scaffold is the product, and its users are end users. Claude Code and OpenAI Codex are harnesses; LangChain and the OpenAI Agents SDK are frameworks. The distinction matters for evaluation, because a benchmark score attributed to a model was in fact produced by a specific harness.

Frameworks are best distinguished by the problem they solve rather than by feature checklists, since nearly all of them now support tool calling, structured output, streaming, and tracing. The useful questions are: does it own control flow or hand it to you, does it persist state across process restarts, does it target one provider or many, and is it still receiving feature work.

Scroll sideways for more →

Two patterns in that table are worth naming. First, consolidation: Microsoft ran two agent frameworks with overlapping scope for two years before merging them, and the merge retired the research-originated one. AutoGen's maintainer wrote that it "will still be maintained, it has a stable API and will continue to receive critical bug fixes and security patches, but we will not be adding significant new features to it."^[40] Second, the code-agent versus tool-call-agent split: smolagents and similar designs let the model express actions as executable code, which composes loops and conditionals naturally at the cost of requiring a sandbox, while JSON tool calling constrains the action space but needs explicit orchestration for anything beyond a sequence.

The general trend has been toward thinner frameworks. When models could not reliably emit structured calls, framework code carried the burden; once they could, the durable value moved to persistence, permissioning, observability, cost control, and evaluation. Overviews such as Best AI Agent Frameworks track the field, but any such list dates quickly, and a framework's maintenance status is more predictive of production risk than its feature count.

Research systems occupy a third category. Magentic-One, released by Microsoft Research in November 2024, used an orchestrator that "plans, tracks progress, and re-plans to recover from errors" while directing specialized agents for web browsing, file navigation, and code execution.^[41] It was published as a generalist multi-agent system with results on GAIA, AssistantBench, and WebArena, and its ledger-based orchestration influenced later designs, but it was a research artifact rather than a supported product.

State, memory, and adaptation

Agent state can include the current conversation, a structured task record, intermediate files, tool results, and durable memories from prior episodes. These stores have different consistency and privacy requirements. Working state supports the present step. Episodic state records past trajectories. Semantic state stores extracted facts or summaries. Procedural state stores reusable actions or code.

Generative Agents implemented a memory stream, retrieval, reflection, and planning for characters in a simulated social environment.^[10] Reflexion stored verbal feedback from previous attempts in an episodic buffer instead of updating model weights.^[11] Voyager combined an automatically generated curriculum with a library of executable skills and feedback-driven program revision in Minecraft.^[13] Each demonstrated a particular memory mechanism in a particular environment. None establishes that unbounded memory or self-written summaries are generally accurate.

Agent memory differs from model training. Adding a note to a prompt or database changes the context presented on later calls; it does not ordinarily update model parameters. Reflexion's phrase "verbal reinforcement learning" describes its feedback-and-memory procedure, not conventional parameter learning.^[11] In Reinforcement learning, an agent learns behavior from interaction in relation to a reward signal, commonly by updating a value function or policy over experience.^[14] A deployed language-model agent may use a fixed model and never learn in this sense.

Memory creates its own failure modes. Summaries can omit qualifications, retrieval can surface irrelevant episodes, and obsolete instructions can outlive the conditions under which they were written. Data from one user or tenant can leak into another if namespaces and access controls are wrong. Durable stores should therefore record provenance, timestamps, ownership, and retention policy. High-impact facts may require confirmation from a source of record rather than recall from a generated summary.

Memory is also an attack surface, and the addition of durable stores changed its severity. A prompt injection that only affects one turn ends when the session ends. One that writes a poisoned instruction into long-term memory persists across sessions and may be retrieved in a context where the original untrusted source is no longer visible. Any durable store an agent can write to should therefore be treated as untrusted input when it is read back, with the same provenance labeling applied to retrieved content generally.

Multi-agent coordination

A Multi-agent system contains multiple interacting agents, which may cooperate, compete, negotiate, or observe one another. In language-model systems, the components may use the same underlying model with different instructions, or different models and tools. Calling several copies of one model does not by itself produce independent evidence because they can share training data, prompts, and failure tendencies.

Coordination designs include a central supervisor, a pipeline of specialized roles, a shared workspace, peer-to-peer messages, voting, and debate. ChatDev studied role-specialized language-model agents communicating across design, coding, and testing phases of selected software-development tasks.^[15] Multiagent Debate reported improvements on the mathematical, strategic, and factual tasks in its experiments.^[16] Those results are task- and protocol-specific; they do not show that adding agents always improves accuracy.

Multiple agents also introduce more interfaces where information can be lost or distorted. A coordinator may assign the wrong subtask, agents may disagree about shared state, messages may omit dependencies, and a verifier may accept an artifact it did not actually test. A 2025 study of traces from selected multi-agent language-model systems organized observed failures into system-design, inter-agent-misalignment, and task-verification categories, and found that benchmark gains were often limited in the systems examined.^[17]

A practical distinction has emerged between two uses of the word. Multi-agent as parallelism, in which one operator runs several subagents to divide context or search breadth, is a context-management technique and is usually invisible to the user; the operator controls every component and can verify results centrally. Multi-agent as a market, in which agents built and operated by different parties transact, is what A2A and payment protocols anticipate, and it raises problems the first case does not: identity, authorization, incentive compatibility, and the possibility that a counterparty is adversarial. Magentic Marketplace, a simulated two-sided market published in October 2025 in which assistant agents represented consumers and service agents represented competing businesses, examined the second case. Its authors reported that frontier models "can approach optimal welfare, but only under ideal search conditions," that "performance degrades sharply with scale," and that "all models exhibit severe first-proposal bias, creating 10-30x advantages for response speed over quality," alongside vulnerability to manipulation.^[61] Results from a simulated market do not transfer directly to a real one, but they identify failure modes that a purely technical protocol does not address.

Agent orchestration should therefore specify ownership of each artifact, allowed communication paths, conflict resolution, time and cost budgets, and an authoritative completion test. Parallel work is valuable when subtasks are genuinely separable. It can be counterproductive when every component repeatedly summarizes the same uncertain evidence or when no component has authority to resolve contradictions.

Evaluation

Agent evaluation measures a system operating through time, not just a model response. The test environment must define its initial state, available actions, hidden information, success condition, time limit, and reset behavior. The evaluator should distinguish task success from partial progress, policy compliance, and absence of harm.

Published benchmarks

Benchmarks cover different slices of this problem. The table lists environments with published methodology; headline figures are those reported by the benchmark authors at publication and are not comparable across rows.

Scroll sideways for more →

These benchmarks are not interchangeable. A system that edits code is not thereby validated for financial transactions, web navigation, or robotic control. Benchmark results also depend on the model version, scaffold, prompts, tool implementations, environment snapshot, attempt budget, and scoring code. Current leaderboard positions are volatile and should not be treated as enduring properties of a named product.

Contamination

Static benchmarks built from public data age badly. The problem has two forms. Training-set contamination occurs when the evaluation items were in the model's pretraining corpus, so recall substitutes for capability. Environment contamination is specific to agents: the agent operates inside a live environment that may itself contain the answer, through repository history, issue comments, or internet access.

Both have been measured on SWE-bench, the most influential agent benchmark. A 2024 audit found that 32.67% of successful SWE-agent patches involved solutions provided directly in the issue report or comments, that 31.08% of passing patches were suspect because of weak tests, and that removing problematic instances dropped one agent's resolution rate from 12.47% to 3.97%; the same audit noted that over 94% of the issues predated the evaluated models' knowledge cutoffs.^[48] SWE-rebench responded by building a continuously refreshed task pipeline with contamination tracked against model release dates, and reported that "performance of some language models might be inflated due to contamination issues."^[47]

Environment contamination persisted after the training-data problem was recognized. In June 2026 Cursor reported that on SWE-bench Pro, 63% of one frontier model's successful resolutions retrieved an existing fix rather than deriving one, and that sealing git history and restricting internet access dropped that model from 87.1% to 73.0% and its own model from 74.7% to 54.0%.^[51] These are vendor-published figures from a company that ships a competing model, and should be read as such, but the mechanism is straightforward and the benchmark maintainers made corresponding changes to strip future git history from environment images.^[51] The general lesson is that any benchmark whose tasks are drawn from public repositories must control what the agent can reach at evaluation time, not only what the model saw during training. See benchmark contamination for the broader problem.

Reward hacking

Reward hacking, also discussed as specification gaming, is distinct from contamination: the agent does not retrieve the answer, it satisfies the scorer without performing the task. In agent evaluation this is unusually easy, because grading is automated and the agent frequently has write access to the environment that contains the grader.

Published work has now measured it directly. BenchJack, an automated red-teaming system applied to 10 popular agent benchmarks spanning software engineering, web navigation, desktop computing, and terminal operations, "synthesizes reward-hacking exploits that achieve near-perfect scores on most of the benchmarks without solving a single task, surfacing 219 distinct flaws across the eight classes" its authors catalogued; an iterative patching pipeline reduced the hackable-task ratio from near 100% to under 10% on four of them.^[49] A separate 2026 benchmark constructed tasks with deliberate shortcut opportunities and evaluated 13 frontier models, reporting exploit rates from 0% to 13.9%, a large gap between a base model and its reinforcement-learning-trained sibling (0.6% against 13.9%), and the finding that 72% of reward-hacking episodes included explicit chain-of-thought rationale, meaning models often framed the exploit as legitimate problem solving.^[50] Environmental hardening reduced exploit rates by 5.7 percentage points in that study without degrading task success.^[50]

Three consequences follow for anyone reading agent scores. A score is a property of a benchmark implementation as much as of an agent, so a benchmark that has not been adversarially audited should be assumed exploitable. Reward hacking is not confined to reinforcement-learning training loops; it appears at evaluation time in models trained with it. And a trajectory that ends in a passing grade is not evidence of a correct solution, which is why trajectory review and independent verification remain necessary. The dedicated page on agent benchmark reward hacking collects specific cases.

Designing an evaluation program

A useful evaluation program includes:

• deterministic unit tests for parsers, permissions, and tool adapters;
• scenario tests for common and difficult task paths;
• repeated trials to measure variance and consistency;
• held-out environments and temporal splits where leakage is plausible;
• fault injection for unavailable tools, malformed results, timeouts, and partial writes;
• adversarial tests for untrusted content and unauthorized goals;
• an adversarial audit of the scorer itself, on the assumption that it can be gamed;
• human review of sampled trajectories and ambiguous failures;
• measurement of task success, policy violations, unsafe actions, interventions, latency, model calls, and external cost.

The evaluation unit should be the complete versioned system. Reporting only the underlying model hides the effects of prompts, interfaces, memory, and control code. An Agent evaluation should also preserve failed trajectories because aggregate success rates do not reveal whether failures were harmless refusals, recoverable misunderstandings, or irreversible actions.

Reliability, security, and oversight

Agent failures include incorrect plans, fabricated facts, wrong tool selection, malformed arguments, repeated side effects, premature termination, and failure to recognize completion. A model Hallucination becomes operationally more consequential when the system can act on it. Long trajectories can amplify small errors as generated state is reused as evidence for later decisions.

Prompt injection and the confused deputy

Security risks arise because agent inputs can combine trusted instructions with untrusted data. In Prompt injection, text attempts to alter the model's behavior. Indirect prompt injection places such text in content retrieved from websites, messages, or documents.

The structural problem is old and has a name in security engineering: the confused deputy, a privileged program tricked into misusing its authority on behalf of a less privileged party. An agent is a near-perfect confused deputy. It holds the user's credentials, it reads content the user did not write, and it cannot reliably tell an instruction from data because both arrive as tokens in the same context. No amount of instructing the model to ignore embedded commands changes that, because the instruction and the attack occupy the same channel.

A useful practical formulation is the "lethal trifecta" described by Simon Willison in June 2025: an agent is exposed when it combines access to private data, exposure to untrusted content, and the ability to communicate externally.^[58] Any two are usually manageable. All three mean a single poisoned document can cause exfiltration, and the exfiltration channel need not be an obvious one, since a rendered image URL or a clickable link suffices.^[58]

Research environments have measured the attack surface. ToolEmu used an emulated sandbox and automated evaluation to search for risky behavior across high-stakes tool scenarios, while noting that emulation and automated judgments themselves required human validation.^[25] InjecAgent evaluated indirect injection attacks against tool-integrated agents across a defined collection of tools and attack intentions.^[26] AgentDojo provided an extensible environment for testing agent tasks, prompt-injection attacks, and defenses over untrusted data.^[27] These studies demonstrate important attack surfaces, not universal attack rates for every model and deployment.

Documented incidents

The transition from research demonstration to production incident is documented. EchoLeak, assigned CVE-2025-32711, was an indirect prompt injection in Microsoft 365 Copilot that "enabled remote, unauthenticated data exfiltration via a single crafted email" with no user interaction. A published analysis describes an attack chain that evaded Microsoft's cross-prompt-injection classifier, circumvented link redaction using reference-style Markdown, exploited auto-fetched images, and abused a proxy permitted by the content security policy, achieving "full privilege escalation across LLM trust boundaries."^[59] The authors' proposed mitigations are architectural rather than model-level: prompt partitioning, input and output filtering, provenance-based access control, and stricter content security policies.^[59]

The case is instructive because every individual defense was present and the composition still failed. A classifier reduced the probability of injection but did not eliminate it; link redaction blocked one exfiltration path but not another; the content security policy allowed a first-party domain that could be repurposed. Defense in depth against injection only works if each layer fails independently, and layers that all depend on the model's judgment do not.

Controls

Security controls should be enforced outside the model whenever possible. Common controls include:

• granting each tool and task the minimum permissions and duration needed;
• separating read, draft, approve, and commit operations;
• isolating code execution, files, credentials, and network destinations;
• treating tool output and retrieved content as untrusted data rather than higher-priority instructions;
• validating arguments against policy and current state before execution;
• requiring confirmation for high-impact, ambiguous, or irreversible actions;
• using transactional writes, idempotency keys, and rollback where available;
• imposing limits on steps, time, tokens, spending, and retries;
• restricting outbound network destinations to an allowlist, since exfiltration requires an egress path;
• recording actions, tool results, approvals, and policy decisions in tamper-resistant logs;
• providing a reliable stop mechanism and a defined incident-response path.

Sandboxing deserves particular attention for code-executing agents, which is now most of them. A sandbox that isolates the file system but not the network leaves the exfiltration channel open. A sandbox that isolates both but runs with the user's cloud credentials mounted has not isolated the thing that matters. The useful question is not whether execution is sandboxed but what an attacker who fully controls the agent's output could reach.

A more ambitious line of defense treats the problem as an information-flow question rather than a prompting question. CaMeL, described in a 2025 paper from Google DeepMind, separates a privileged model that plans from a quarantined model that reads untrusted data, and executes the resulting plan in a custom interpreter that tracks provenance and enforces policies before each tool call, providing guarantees without modifying the underlying model.^[57] The authors report that it effectively solves the AgentDojo security evaluation while constraining unintended actions and data exfiltration.^[57] The design cost is real: it requires the plan to be expressible before untrusted data is read, which does not fit every task. As of August 2026 no comparable approach has been adopted as a default in widely used agent frameworks, and prompt injection remains unsolved in the general case.

Circuit breakers and other runtime interventions address a related but distinct problem, interrupting a model mid-generation when internal representations indicate harmful trajectories, rather than filtering the inputs or outputs. Guardrails libraries occupy the same layer and share the same limitation: a check implemented as another model call inherits the failure modes of model calls.

Oversight

Human-in-the-loop control is effective only when the reviewer receives enough context, has time and authority to intervene, and is asked at a meaningful decision point. Asking a person to approve a dense stream of low-level actions can produce routine confirmation rather than oversight. The system should identify which decisions require judgment and present the proposed action, supporting evidence, expected effect, and available alternatives.

The approval-fatigue problem grew worse as horizons lengthened. An agent that runs for minutes can reasonably ask before every write. An agent that runs for hours and makes hundreds of tool calls cannot, which is why permission models moved toward pre-authorized scopes: the operator approves a class of actions in advance, and the agent proceeds within it. That shifts the human judgment earlier and makes the scope definition the safety-critical artifact.

The NIST AI Risk Management Framework describes risk management as a continuous process organized around govern, map, measure, and manage, and calls for clearly defined human roles, monitoring, testing, and mechanisms for intervention.^[28] Its Generative AI Profile applies that framework to generative systems and emphasizes evaluation, documentation, incident disclosure, content provenance, privacy, and information-security risks.^[29] These are risk-management resources, not certifications that a particular agent is safe. Broader AI governance obligations may also apply depending on jurisdiction and sector.

AI safety and AI Alignment cover broader questions about harmful behavior and the relation between system objectives and intended values. Agentic misalignment concerns cases in which goal-directed behavior conflicts with operator intent. For deployed agents, those questions meet ordinary security and reliability engineering at concrete boundaries: who set the objective, what actions are authorized, what evidence can change the plan, and who can stop execution.

Deployment evidence

Claims about agent productivity are made far more often than they are measured. The published evidence is thin, mostly recent, and mostly less favorable than product descriptions. It is also methodologically difficult: the counterfactual is what the same person or organization would have achieved without the agent, which is rarely observed.

The strongest study design available is a randomized controlled trial. METR ran one with 16 experienced open-source developers on 246 real tasks in repositories they already maintained, using tools available between February and June 2025. Developers took 19% longer to complete tasks when allowed to use AI tools, while forecasting beforehand and estimating afterwards that the tools had sped them up by roughly 20%.^[54] The gap between measured and perceived effect is the study's most transferable result, because self-reported productivity gains are the basis of most published adoption claims.

That finding should not be over-extended, and METR itself has qualified it. In February 2026 the organization reported that a follow-up study found a preliminary 18% reduction in completion time among the original participants, with a confidence interval spanning -38% to +9%, and simultaneously announced that it was redesigning the experiment because the design had become unworkable: developers increasingly declined to participate in conditions requiring them to work without AI, and "30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI." METR concluded that "our data is only very weak evidence for the size of this increase."^[55] The honest reading as of August 2026 is that the early-2025 slowdown was real for that population and toolset, that the effect has probably moved, and that selection effects now make randomized measurement of experienced developers genuinely hard.

Benchmark environments built to resemble real work point the same direction. In TheAgentCompany, a simulated software company with browsing, coding, and colleague-chat interfaces, "the most competitive agent can complete 30% of tasks autonomously."^[43] In CRMArena-Pro, leading agents reached about 58% single-turn success and about 35% in multi-turn settings, and showed "near-zero inherent confidentiality awareness," with targeted prompting improving confidentiality at the cost of task performance.^[45] In Windows Agent Arena the baseline agent's 19.5% sat against 74.5% for an unassisted human.^[42] These are not measurements of deployed products, but they bound expectations: an agent that completes a third of realistic office tasks unaided is a useful assistant under supervision and a poor substitute for an unsupervised worker.

Organizational evidence is weaker still, and mostly consists of surveys and analyst forecasts rather than measurement. Gartner predicted in June 2025 that more than 40% of agentic AI projects would be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls, and reported that in a January 2025 poll of 3,412 respondents only 19% said their organization had made significant investments in agentic AI.^[56] A forecast is not a finding, and analyst survey populations are self-selected, so the value of such statements is as a description of prevailing conditions rather than evidence about capability.

Three regularities hold across the available evidence. Success rates on realistic, multi-step, long-horizon work remain far below success rates on curated single-step tasks. Perceived benefit exceeds measured benefit where both have been collected. And the categories where agents demonstrably sell, coding first among them, are those with cheap automatic verification, bounded environments, reversible errors, and technically capable supervisors. Extrapolating from those categories to ones lacking those properties is not supported by anything published so far.

Deployment and reproducibility

A deployment specification should begin with a task envelope. It states permitted users, data, tools, environments, consequences, and expected uncertainty. It also defines out-of-scope requests, approval thresholds, recovery procedures, and the authoritative source used to determine completion. This is more informative than assigning the system a single "autonomy level."

Staged deployment can move from offline traces to simulated tools, read-only operation, reversible writes, supervised production, and only then broader authority if evidence supports it. Monitoring should compare actual tool use and outcomes with the task envelope. Unexpected action sequences, repeated failures, permission denials, cost spikes, or shifts in human intervention rates can indicate a model, prompt, environment, or integration change.

Reproducible reporting should identify the model and version, scaffold code, System prompt, tool schemas and versions, protocol revisions in use, environment snapshot, memory policy, decoding settings, step and budget limits, approval rules, retry logic, and evaluator. It should report the number of trials and the distribution of outcomes, not only the best run. A Model card can document the underlying model, but an agent also needs system-level documentation covering tools, permissions, data flows, and operational controls.

Version pinning deserves specific mention because the surface an agent depends on now changes underneath it. A model is deprecated on a provider's schedule, a framework reaches a major version with a new control-flow API, a protocol revision removes a feature after its deprecation window, and a remote tool server changes a schema without notice. An agent that was evaluated once and left running is not the same system a year later, which is an argument for continuous evaluation against a fixed internal suite rather than one-time acceptance testing.

AI agents are therefore best understood as bounded socio-technical systems. Their useful capabilities come from combining models with interfaces, state, tools, feedback, and people. The same composition creates failure and security paths that a model-only evaluation cannot observe. Claims about capability, reliability, or autonomy are meaningful only when tied to a versioned system, a defined environment, and a stated level of authority.

References

1. ^Michael Wooldridge and Nicholas R. Jennings, "Intelligent Agents: Theory and Practice," The Knowledge Engineering Review, 1995. doi.org/...S0269888900008122
2. ^Reiichiro Nakano et al., "WebGPT: Browser-assisted question-answering with human feedback," 2021. arxiv.org/...2112.09332
3. ^Brian Ichter et al., "Do As I Can, Not As I Say: Grounding Language in Robotic Affordances," Proceedings of the 6th Conference on Robot Learning, 2023. proceedings.mlr.press/...ichter23a
4. ^Shunyu Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models," International Conference on Learning Representations, 2023. openreview.net/forum
5. ^Timo Schick et al., "Toolformer: Language Models Can Teach Themselves to Use Tools," Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...a906-Abstract-Conference
6. ^Minghao Li et al., "API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs," Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. aclanthology.org/2023.emnlp-main.187
7. ^Shunyu Yao et al., "Tree of Thoughts: Deliberate Problem Solving with Large Language Models," Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...7aaef84ed5ac703-Abstract
8. ^Karthik Valmeekam et al., "On the Planning Abilities of Large Language Models: A Critical Investigation," Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...f880-Abstract-Conference
9. ^John Yang et al., "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering," Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...1e7c-Abstract-Conference
10. ^Joon Sung Park et al., "Generative Agents: Interactive Simulacra of Human Behavior," Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023. doi.org/...3586183.3606763
11. ^Noah Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning," Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...0e90-Abstract-Conference
12. ^Jie Huang et al., "Large Language Models Cannot Self-Correct Reasoning Yet," International Conference on Learning Representations, 2024. openreview.net/forum
13. ^Guanzhi Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models," Advances in Neural Information Processing Systems 36, 2023. nips.cc/...79179
14. ^Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, second edition, 2018. incompleteideas.net/...the-book-2nd
15. ^Chen Qian et al., "ChatDev: Communicative Agents for Software Development," Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024. aclanthology.org/2024.acl-long.810
16. ^Yilun Du et al., "Improving Factuality and Reasoning in Language Models through Multiagent Debate," Proceedings of the 41st International Conference on Machine Learning, 2024. proceedings.mlr.press/...du24e
17. ^Mert Cemri et al., "Why Do Multi-Agent LLM Systems Fail?," Advances in Neural Information Processing Systems 38, 2025. proceedings.neurips.cc/...ets_and_Benchmarks_Track
18. ^Xiao Liu et al., "AgentBench: Evaluating LLMs as Agents," International Conference on Learning Representations, 2024. openreview.net/forum
19. ^Shuyan Zhou et al., "WebArena: A Realistic Web Environment for Building Autonomous Agents," International Conference on Learning Representations, 2024. openreview.net/forum
20. ^Gregoire Mialon et al., "GAIA: A Benchmark for General AI Assistants," International Conference on Learning Representations, 2024. openreview.net/forum
21. ^Carlos E. Jimenez et al., "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?," International Conference on Learning Representations, 2024. openreview.net/forum
22. ^Tianbao Xie et al., "OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments," Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...ets_and_Benchmarks_Track
23. ^Shunyu Yao et al., "Tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains," International Conference on Learning Representations, 2025. openreview.net/forum
24. ^Chang Ma et al., "AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents," Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...ets_and_Benchmarks_Track
25. ^Yangjun Ruan et al., "Identifying the Risks of LM Agents with an LM-Emulated Sandbox," International Conference on Learning Representations, 2024. proceedings.iclr.cc/...d1c5f04-Abstract-Conference
26. ^Qiusi Zhan et al., "InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents," Findings of the Association for Computational Linguistics: ACL 2024, 2024. aclanthology.org/2024.findings-acl.624
27. ^Edoardo Debenedetti et al., "AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents," Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...ets_and_Benchmarks_Track
28. ^Elham Tabassi, Artificial Intelligence Risk Management Framework (AI RMF 1.0), National Institute of Standards and Technology, 2023. doi.org/...NIST.AI.100-1
29. ^Chloe Autio et al., Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, National Institute of Standards and Technology, 2024. doi.org/...NIST.AI.600-1
30. ^Anthropic, "Building effective agents," 19 December 2024. anthropic.com/...building-effective-agents
31. ^Model Context Protocol maintainers, "MCP joins the Agentic AI Foundation," 9 December 2025. blog.modelcontextprotocol.io/...ntic-ai-foundation
32. ^The Linux Foundation, "Linux Foundation Announces the Formation of the Agentic AI Foundation (AAIF), Anchored by New Project Contributions Including Model Context Protocol (MCP), goose and AGENTS.md," 9 December 2025. linuxfoundation.org/...f-the-agentic-ai-foundation
33. ^Model Context Protocol maintainers, "The 2026-07-28 Specification," 28 July 2026. blog.modelcontextprotocol.io/...2026-07-28
34. ^Model Context Protocol documentation, "Versioning," accessed 1 August 2026. modelcontextprotocol.io/...versioning
35. ^Google Developers Blog, "Google Cloud donates A2A to Linux Foundation," June 2025. developers.googleblog.com/...a-to-linux-foundation
36. ^Google Open Source Blog, "A year of open collaboration: Celebrating the anniversary of A2A," April 2026. opensource.googleblog.com/...he-anniversary-of-a2a
37. ^AG-UI Protocol project, ag-ui-protocol/ag-ui repository, accessed 1 August 2026. github.com/...ag-ui
38. ^LangChain, "LangChain and LangGraph Agent Frameworks Reach v1.0 Milestones," 22 October 2025. langchain.com/...langchain-langgraph-1dot0
39. ^Yina Arenas, "Introducing Microsoft Agent Framework," Microsoft Azure Blog, 1 October 2025. azure.microsoft.com/...g-microsoft-agent-framework
40. ^Eric Zhu, AutoGen maintenance-mode announcement, microsoft/autogen GitHub Discussion #7066, 2 October 2025. github.com/...7066
41. ^Adam Fourney et al., "Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks," 2024. arxiv.org/...2411.04468
42. ^Rogerio Bonatti et al., "Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale," 2024. arxiv.org/...2409.08264
43. ^Frank F. Xu et al., "TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks," Advances in Neural Information Processing Systems 38, Datasets and Benchmarks Track, 2025. arxiv.org/...2412.14161
44. ^Tianci Xue et al., "An Illusion of Progress? Assessing the Current State of Web Agents," 2025. arxiv.org/...2504.01382
45. ^Kung-Hsiang Huang et al., "CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions," 2025. arxiv.org/...2505.18878
46. ^Andy K. Zhang et al., "BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems," 2025. arxiv.org/...2505.15216
47. ^Ibragim Badertdinov et al., "SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents," 2025. arxiv.org/...2505.20411
48. ^Reem Aleithan et al., "SWE-Bench+: Enhanced Coding Benchmark for LLMs," 2024. arxiv.org/...2410.06992
49. ^Hao Wang et al., "Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack," 2026. arxiv.org/...2605.12673
50. ^Kunvar Thaman, "Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use," 2026. arxiv.org/...2605.02964
51. ^Cursor, "Reward hacking is swamping model intelligence gains," 25 June 2026. cursor.com/...reward-hacking-coding-benchmarks
52. ^METR, "Measuring AI Ability to Complete Long Tasks," 19 March 2025. metr.org/...ring-ai-ability-to-complete-long-tasks
53. ^METR, "Time Horizon 1.1," 29 January 2026. metr.org/...2026-1-29-time-horizon-1-1
54. ^Joel Becker et al., "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," 2025. arxiv.org/...2507.09089
55. ^METR, "We are Changing our Developer Productivity Experiment Design," 24 February 2026. metr.org/...2026-02-24-uplift-update
56. ^Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027," 25 June 2025. gartner.com/...cts-will-be-canceled-by-end-of-2027
57. ^Edoardo Debenedetti et al., "Defeating Prompt Injections by Design," 2025. arxiv.org/...2503.18813
58. ^Simon Willison, "The lethal trifecta for AI agents: private data, untrusted content, and external communication," 16 June 2025. simonwillison.net/...the-lethal-trifecta
59. ^Pavan Reddy and Aditya Sanjay Gujral, "EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System," 2025. arxiv.org/...2509.10540
60. ^Microsoft, FY26 second-quarter earnings call remarks by Satya Nadella, 28 January 2026. microsoft.com/...earnings-fy-2026-q2
61. ^Gagan Bansal et al., "Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets," 2025. arxiv.org/...2510.25779