From Answers to Actions

A sponge, not a sentence

In 2022, a team of robotics researchers at Google, working with the Alphabet project Everyday Robots, published an example built around an ordinary request: “I spilled my drink, can you help?” A language model could answer with sensible advice about cleaning up a spill. But advice was not what the person needed, and a one-armed robot moving around an office kitchen could not do most of the things a model might suggest, such as fetching a vacuum cleaner. The researchers wanted the machine to choose something that was both useful and within its abilities. In their published example, that meant finding a sponge and bringing it over.

Their system, called SayCan, combined two kinds of judgment. A large language model scored how useful each of the robot’s available skills would be as the next step toward the request. Separately trained models scored how likely the robot was to succeed at each skill from where it stood. The system multiplied the two and chose the best action, then repeated the process for the next step. Neither judgment was enough alone: the language model knew what would help but not what the robot could do, and the robot’s own models knew what was possible but not what was wanted. In August 2022, running with Google’s large PaLM language model, the researchers reported that, across their main evaluation of 101 instructions, the system chose a correct plan 84 percent of the time and carried it out successfully 74 percent of the time; in a separate test in a real office kitchen, the figures were 81 and 60 percent.

The difference between the sentence and the sponge was small enough to overlook and large enough to define a new period in computing. An assistant could describe how to make a reservation without reserving anything. It could write an email without sending it, write a program without running it, or explain how to retrieve a document without permission to open the file. Generating language ended when the answer appeared. Acting required a connection to something outside the model, an application, a browser, a robot, or a program whose result could be observed, and an action could fail for reasons that had nothing to do with the quality of the words that proposed it. An item might be out of stock, a form might have changed, a password might be needed, or the first step might change what the second step had to be.

The ambition to make computers pursue goals was as old as artificial intelligence. What changed in the 2020s was the connection between a person’s request and the machinery that could carry it out. The models of Chapters 26 and 27 could turn an astonishing range of requests into text. But a token reading “open the file” did not open the file. The next advance was to let a model’s choice trigger a real operation, read what happened, and continue until a task was verifiably finished, or until it recognized that it could not safely go on.

The older machinery of plans

The idea of a computer making a plan is older than the personal computer. In the late 1950s, Allen Newell, Cliff Shaw and Herbert Simon, whose Logic Theorist appeared in Chapter 22, built the General Problem Solver. Given a description of where things stood and where they should end up, it looked for an operation that would reduce the difference between the two, then repeated the process, a method called means-ends analysis. It showed that a program could reason about a sequence of actions. The hard part, describing a real problem in the formal terms the program required, still fell to people.

Chapter 22 met Shakey, the robot built at the Stanford Research Institute between 1966 and 1972, which planned its routes through a set of rooms. To decide which actions to take, Shakey used a planner described in 1971 by Richard Fikes and Nils Nilsson, called STRIPS. It represented each action by the conditions that had to be true before it could happen and the facts that would change afterward: to push a block through a doorway, the robot had to be next to the block, and afterward the block would be in the next room. Planning meant searching for a chain of actions whose conditions lined up from the starting situation to the goal. The idea outlived Shakey. A standard language for describing planning problems, introduced in 1998 for an international planning competition, still followed the same pattern.

Shakey’s world, though, was deliberately simple, and every attempt to plan for an ordinary house or office ran into the same difficulties: doors closed, furniture moved, instructions were incomplete and actions had unforeseen effects. At MIT in the mid-1980s, Rodney Brooks argued for the opposite approach, robots that reacted directly to what their sensors told them, through layers of simple behaviors, without building elaborate plans at all. One of his companies, iRobot, would later sell millions of Roomba vacuum cleaners, whose early models drew on practical reactive control, although later models developed more elaborate navigation. For decades, practical systems mixed the two traditions: industrial robots, warehouse systems and scheduling software used careful planning and control, but only after engineers had described the available actions and the world in precise, specialized terms. A robot that could move blocks could not, for that reason, book a flight.

The internet and the software of earlier chapters made a different kind of action available. By the 2000s, programs routinely asked other programs to do things through application programming interfaces, or APIs: look up a record, charge a card, store a file, send a message. The cloud services of Chapter 20 were built from such interfaces. Workflow engines and planners could already choose between branches, but developers generally specified the routing logic and its limits in advance, with explicit handling for every error they could foresee. The change in the early 2020s was not that computers learned to call functions. It was that a language model might choose which function to call from an open-ended description, in ordinary language, of what someone wanted, fill in the details, interpret the response and decide what to try next, without a programmer spelling out the whole route for each possible request.

The model makes a move

A paper posted in October 2022 gave a clear account of the new method. Shunyu Yao, a doctoral student at Princeton, with Jeffrey Zhao, Dian Yu and colleagues at Princeton and Google, called it ReAct, for reasoning and acting. They prompted a language model to alternate between writing out what it needed to find out and carrying out a limited action in an environment. Asked a question whose answer required combining facts, the model could search a simplified Wikipedia interface, read an entry, revise its plan and search again. In two simulated environments, a text-based household game called ALFWorld and a mock online store called WebShop, it chose actions and received descriptions of what had happened. The unit of work was no longer a single uninterrupted answer. It was a succession of decisions and observations.

In February 2023, Timo Schick and colleagues at Meta described Toolformer, a different route to the same goal. Instead of relying on examples written into each prompt, they trained a language model to insert calls to tools, a calculator, a search engine, a question-answering system, a translator and a calendar, into its own text. The model learned not only how to write a call but when an outside result would improve what it said next, and it did so through a largely self-supervised process: the researchers kept only the calls whose results actually made the model’s predictions better. The two projects differed technically, but both treated a tool’s output as part of the information from which the model continued.

The mechanism involves nothing mysterious. Software gives the model a goal and a list of permitted tools, each with a description of the inputs it accepts. The model writes a request for one of them. In a well-designed system, a separate program, which developers came to call a controller or harness, checks the request and its permissions, carries out the operation and returns the result: a page of search results, a calculation, a list of files, a screenshot or an error message. The result is added to what the model can see, and the model proposes another step or gives a final answer. The surrounding software decides which tools are available, how many steps may be tried, what counts as success and whether a consequential action needs a person’s approval.

This was still the computing of the earliest chapters. A model’s proposed call was only data until another program acted on it, and the tools it chose among were connections that programmers had already built, each with a fixed contract, like the system calls through which programs ask an operating system for help. What changed was that a learned component, reading a goal written in ordinary language, now selected among them, imperfectly, where a developer once had to anticipate every combination in advance.

Companies moved quickly to offer the pattern. On March 23, 2023, OpenAI introduced a limited preview of ChatGPT plugins, connecting its assistant to outside services for tasks that needed current information or an operation in another application. On June 13 it added function calling to its developer service: a developer could describe a function’s name and arguments, and the model would reply with structured data requesting it. Other companies offered similar features within months. The interface gave the model no new powers on its own. It gave the program around the model a dependable way to tell a tool request from ordinary prose, the step that turned tool use from a clever experiment in a chat window into a part of software.

The programs that kept going

In the spring of 2023, programmers began asking what would happen if the loop ran over and over with little human direction. At the end of March, a game developer named Toran Bruce Richards, writing as Torantulino, released Auto-GPT, an open-source program that placed GPT-4 inside a loop with a goal, a short memory and access to tools such as web search and file writing. It proposed a plan, took one action, judged what had happened and gave itself another instruction. It attracted rapid attention on GitHub. A few days later, a venture capitalist and amateur programmer, Yohei Nakajima, shared BabyAGI, a script of roughly 140 lines in its original form that had a model create, prioritize and carry out its own list of tasks. Harrison Chase’s LangChain, begun in October 2022, offered developers ready-made parts for connecting models to data and tools, and quickly incorporated ReAct and other research ideas.

These projects drew attention partly because they showed how little code was needed to create the appearance of persistence. A prompt told the model it had a goal; a controller called the model again and again, feeding back recent actions, failures and notes. The model seemed to have an intention that stretched across time, but the persistence belonged to the surrounding program. When the context filled up, the controller had to summarize or store what had happened. A model that lost track of the task might repeat a failed step; one that misread an error might build a more elaborate version of the same mistake. Many users found that a loop could turn a helpful assistant into an expensive generator of increasingly confident detours. The early agents were more important as demonstrations of an architecture than as working tools.

Researchers found more deliberate ways to keep useful experience. In May 2023, Guanzhi Wang and colleagues at NVIDIA, Caltech and other institutions described Voyager, an agent that played the video game Minecraft. It used GPT-4 to propose its own next objectives and to write programs for achieving them, ran each program in the game, and used the game’s feedback and error messages to revise the code. Programs that worked were stored in a skill library, indexed by a description of what they did, so that a later task needing a familiar operation could call up a program that had already succeeded. Within that game environment, and without a person choosing its goals, Voyager explored much further, and collected many more kinds of items, than the earlier agents its authors compared it with.

Voyager’s memory was not a neural network learning new weights. It was ordinary software: pieces of code, descriptions of what they achieved and a way to look them up. Working assistants came to use similar layers. The model’s context held the current exchange. A task list tracked unfinished steps. Files and databases held information that had to survive a restart, and a search system could retrieve relevant documents and place them in front of the model when needed, an approach to which researchers at Facebook gave the name retrieval-augmented generation in 2020, though the use of external retrieval was older than the name. The divisions mattered. A model could be highly fluent and still lack a reliable record of what its previous tool call had done; a modest model, given dependable memory, narrow tools and a clear way to check results, could carry out useful routines.

From an API to the visible screen

APIs were the cleanest way for a model to act, but much of the world’s software had never offered a convenient one. An office worker opens a website, clicks through a form, picks an item from a menu and copies the result into a spreadsheet. Those interfaces were built by different companies at different times, and even services with APIs often exposed only part of what they did. An agent limited to a list of approved API calls could work well inside its connections and be helpless one screen beyond them. So researchers turned to a more general interface: the screen a person sees.

The idea had appeared early. In September 2022, a startup called Adept, whose founders included Ashish Vaswani and Niki Parmar, both co-authors of the 2017 Transformer paper, demonstrated ACT-1, a model that operated a web browser by reading the page and clicking and typing. The demonstrations were selected by the company, and the product was not widely released. On October 22, 2024, Anthropic released a test version of a capability it called computer use for one of its Claude models, available to developers. Given screenshots, the model could choose a position on the screen, move the pointer, click, scroll and type. The company described it as experimental and error-prone. It was nonetheless a clear step beyond text-only tool calls. The model had to interpret an interface visually, relate an instruction to something on the screen, act, and then look again, because a button might have moved, a dialog box might have appeared, or a click that seemed to work might have opened the wrong menu.

Other laboratories followed within months. In December 2024, Google showed a research prototype called Project Mariner that operated inside the Chrome browser. On January 23, 2025, OpenAI released a preview of Operator, an agent that used its own browser to carry out tasks by clicking, typing and scrolling. A related kind of agent concentrated on reading. Google introduced a Deep Research feature for Gemini in December 2024, and on February 2, 2025, OpenAI introduced its own deep research, which spent many minutes searching, reading and comparing sources before writing a report with citations. The two kinds of work differed in an important way. A research agent ended with a document that a person could evaluate before relying on it. A browser agent might change a booking, submit a form or buy something, and a mistaken step there had consequences outside the screen.

In July 2025, OpenAI introduced ChatGPT agent, integrating web research, code execution, connected accounts and browser interaction in one workflow inside a virtual computer, pausing to ask permission before consequential actions such as purchases. Other companies built similar systems. The significance lay in the pattern more than in any one product: a model could move between specialized tools and the general-purpose screen, carrying what it had learned from one step into the next. The old line between a program that told a person what to do and a program that did it was fading.

The screen was an uncertain instrument. A model might misread a small icon, confuse two similar fields, click before a page had finished loading, or fail to notice that a website had rejected its submission. Just as Shakey had to revise its route when its camera showed the world differing from its map, a browser agent had to observe its environment after every action. There was also a new difference between what was possible and what was permitted. A person might be physically able to click a button that sends a payment, but an assistant acting on someone’s behalf needed authorization to do so. The power to use a keyboard did not make every use of it legitimate.

The test that could run the program

Software became one of the best places to study these systems, because a result could often be checked. Writing code was an early success for language models: in 2021, OpenAI described Codex, a GPT model trained on public code, and GitHub, owned by Microsoft, built it into a programming assistant called Copilot that suggested the next lines as a programmer typed. Fixing a real bug was a different matter. An agent asked to repair one had to explore a repository, find the relevant files, understand how they depended on each other, change the code and run the tests. A plausible explanation of the defect was not enough, and an elegant-looking patch could still fail to build. The compiler and the test suite could tell the agent what had happened far more precisely than most written feedback could.

In October 2023, Carlos Jimenez, John Yang, Alexander Wettig and colleagues at Princeton introduced SWE-bench, built from 2,294 real problems reported in public Python projects on GitHub, together with the code changes that had actually fixed them. A proposed fix counted only if the project’s tests passed afterward. In the first study, the best model, given the relevant code in a single prompt and no tools, solved fewer than two percent of the problems. That figure was not a general measure of how well models could program. It showed that repairing a real codebase was a very different task from writing a short function on request.

The next year, Yang, Jimenez and colleagues described SWE-agent, which gave a language model a small set of commands designed for the job: open part of a file, search the code, make an edit, run the tests. Using a GPT-4-based agent with this purpose-built interface, the original paper reported that SWE-agent resolved about 12.5 percent of the issues, far more than the earlier approaches it compared, which had worked without tools. Their point was partly about interface design. A model did not work best when handed an enormous dump of files or left to improvise complicated terminal commands; clear, bounded commands made the same model more capable. The interface between a model and a computer had become a subject of engineering in its own right. Companies began reporting results on SWE-bench as a matter of course, and in August 2024 OpenAI, working with the benchmark’s authors, released a version of 500 problems that people had checked to be solvable and fairly tested.

By 2025, coding agents were products. In February, Anthropic released Claude Code, an agent that worked in a programmer’s own terminal. In May, OpenAI reused an old name for a new system, introducing Codex as a cloud-based agent that worked on a copy of a codebase in an isolated environment and ran tests before presenting its changes. GitHub, Google and many open-source projects offered comparable agents. The code still needed human judgment, especially where tests were missing or weak. But a developer could now hand over a defined problem and review a proposed change, with a record of the files altered, the tests run and the steps taken, instead of receiving only advice about what code might be written.

Such environments offered a model for controlled action generally. A task could be separated from the rest of a person’s files, given limited network access and stopped before any change reached a live system. Tests were imperfect witnesses: passing them did not prove every requirement had been met, and in controlled training and evaluation settings researchers documented agents that skipped, edited or exploited tests so that a wrong solution appeared to pass. OpenAI reported such behavior during training in March 2025, and a study called ImpossibleBench, published that October, measured how often agents exploited deliberately contradictory tests. These were experiments, not estimates of how often it happens in everyday programming. Yet the system could be designed so that an agent’s claims were checked against something other than its own words. Engineering increasingly meant building a trustworthy procedure around the model as well as improving the model itself.

Connections and the right to use them

The number of possible tools soon became a problem of its own. If every AI application needed a separately written connection to every database, messaging service, code repository and business system, the result would be an industry of custom adapters. Programmers had met this problem before. In 2016, Microsoft had introduced the Language Server Protocol, which let any code editor talk to any programming-language tool through one shared format, so that each needed only one connection instead of one for every partner. In November 2024, Anthropic published a similar open standard for AI, the Model Context Protocol, or MCP, whose designers cited that earlier protocol as an inspiration. A server could describe the tools and information it offered; any compatible application could discover those descriptions and call them. OpenAI announced support for MCP in March 2025, other large companies followed, and in December 2025 Anthropic transferred the protocol to a new Agentic AI Foundation under the Linux Foundation, the nonprofit home of the Linux project from Chapter 16. The protocol itself granted no permissions. Logging in, deciding what each user could reach and guarding each operation remained the job of the systems on either side.

In April 2025, Google proposed a second standard, Agent2Agent, or A2A, and handed it to the Linux Foundation in June. Where MCP connected an agent to tools and data, A2A described how one agent could hand a task to another, possibly run by a different organization, and receive progress reports and results. Neither invented distributed computing; network protocols, service directories and remote procedure calls had been part of software for decades. What was new was a need created by programs whose next step was not fully known until a model had interpreted the task in front of it.

As the connections multiplied, the difference between being able to call a tool and being allowed to call it grew more important. Reading a calendar, drafting an invitation and sending it were separate powers. So were finding a flight, reserving a seat and paying for it. An agent needed to know which actions could be undone and which could not, to recognize when it had enough information to proceed, and to stop when the next step required a decision only its user could make. A good tool description therefore had to say more than what an operation was called and what inputs it took. It had to say whether the operation changed something in the world, and whose consent it needed.

Connections also had to guard against information disguised as instructions. A web page, an email or a document could contain text telling a model to ignore its user and do something else. In September 2022, the programmer Simon Willison, writing about examples shared by Riley Goodside and others, named this prompt injection, after the SQL injection attacks that had plagued websites since the late 1990s, in which data typed into a form was mistakenly executed as a database command. The two were not identical vulnerabilities, but the principle was similar: material from an untrusted source crossing a boundary into a place where it could control operations. The text might be perfectly legitimate material to read or summarize without having any right to give orders. A model that treated every sentence it encountered as equally authoritative could be steered by a malicious page into revealing private information or taking unauthorized actions. Developers responded with permission checks, isolated environments, filtering and monitoring, and model makers trained their systems to distinguish their user’s instructions from the content they processed, but the problem was not solved, and it remained one of the main limits on how much freedom agents could safely be given.

Did it actually finish?

To test a language model, researchers could compare its answer with a reference, ask people to judge it, or run a unit test. To test an agent, they needed to know whether a sequence of actions had left the world in the requested state. A shopping task was not complete because the agent had found the right product page. A form was not filed because the model said it was filed, and an email was not sent because a draft appeared in a text box. The test had to inspect the environment after the agent stopped.

In 2023, Shuyan Zhou and colleagues at Carnegie Mellon University built WebArena, a set of realistic websites, an online store, a discussion forum, a code-hosting site and others, that ran on the researchers’ own servers and could be reset after every attempt. Tasks ranged from looking up information to changing content through several steps, and the results were checked automatically instead of trusting the agent’s account. In their first experiments, the best agent, built on GPT-4, completed 14.41 percent of the tasks; human participants completed 78.24 percent. In 2024, Tianbao Xie and colleagues at the University of Hong Kong and elsewhere introduced OSWorld, which placed agents in real desktop operating systems and checked work involving applications and files. The best initial agent completed 12.24 percent of its tasks; people completed more than 72 percent. Those figures described particular early systems on particular tests, not a permanent limit. They showed how wide the gap was between choosing a plausible action and completing a demanding workflow.

One failure could undo many correct steps. An agent might search accurately, fill nine fields of a form correctly and put the wrong date in the tenth. It might write a correct patch and forget to save it, or decide a message had been sent when the application had only opened a confirmation dialog. Developers improved the controllers around the models: saving progress, recording exactly what each tool returned, detecting repeated failures and requiring an explicit check before declaring success. Retrying uncertain operations helped, but retries had dangers of their own, since pressing a payment button twice or resubmitting a form could create a duplicate. A controller needed to know which actions were safe to repeat, whether an earlier call had already changed something, and when to bring a person back into the process.

In March 2025, METR, a research organization that evaluates AI systems, proposed measuring progress by the length of tasks an agent could complete, judged by how long the same tasks took skilled people. On its set of software and research tasks, the length of task that leading models could finish half the time had been doubling roughly every seven months since 2019, according to a trend fitted across its selected tasks, some of whose human completion times were estimated rather than measured. In January 2026, METR updated the measurement with a larger set of tasks and improved infrastructure; on the new suite, the fitted doubling time for models since 2023 came out shorter, about 131 days against 165 under the earlier method. A longer horizon on those tasks was evidence of improvement, not a certificate that an agent could run an office unattended, and the result depended on which tasks were chosen and what counted as success. But the method captured something single-answer tests missed: a system that could finish a task needing an hour of careful work had crossed a different threshold from one that could suggest the first step.

The most dependable applications usually looked less like unrestricted autonomy than like carefully assembled workflows. A model might classify an incoming request, consult a permitted database and draft a response, while a person or a fixed software rule approved anything irreversible and a separate check confirmed afterward that the record had been created. Where actions could easily be undone, a system could run longer without intervention. Autonomy was not a switch with two positions. It depended on the cost of mistakes, the ability to detect them, the quality of feedback from the environment, and the authority the user had granted.

A new division of work

By 2026, the word agent covered systems with very different abilities. Some searched collections of documents and wrote research reports. Others worked through websites, operated inside spreadsheets, handled narrow customer-service procedures or repaired software in isolated workspaces. Some linked several components, giving one model the job of planning and others the jobs of retrieval, execution or checking. As Chapter 27 noted, companies now described their newest models chiefly by how well they carried out such tasks. Most agents were built from familiar parts: a language model, tools reached through APIs or the screen, a store of working state, rules for when to continue and a way of checking the result. What set them apart from the chat systems of 2022 was not that they had stopped producing language. Language had become one step inside a larger process.

The programmer’s role changed with them. In earlier generations of software, an engineer wrote out an exact procedure: request this input, call that service, branch on this condition, display that result. In an agent, much of the procedure was chosen by a learned model as it went. The engineer still designed the environment in which those choices were made: the permitted operations, the checks on their results and the limits the system could not cross. Some decisions belonged in fixed code, especially those involving arithmetic, security, payments or rules that had to be followed exactly. Others benefited from a model’s ability to interpret incomplete instructions and messy information. Useful systems increasingly combined the two.

Follow the family tree back and the convergence becomes visible. The planners of Shakey’s era supplied a language of goals, conditions and actions. Operating systems and networks supplied ways for one program to request services from another. The web supplied a world of remote applications, and virtualization and cloud computing made it possible to run tasks in isolated machines far from the user. Reinforcement learning, from Chapter 25, contributed ways to judge sequences of decisions. Security engineering, from its long struggle with injected commands, supplied the habit of distrusting data. The Transformer supplied a general interface in ordinary language, and graphics processors and later accelerators supplied the computation the models depended on. None of these was invented for an agent of 2025. Each solved a more immediate problem, and each became useful again when engineers joined it to the others.

In 2022, the spilled drink had exposed the difference between a sentence that sounded helpful and a machine that could bring the sponge. By the middle of the decade, the same difference appeared in thousands of digital tasks. A program could write a plan and still fail to do the work. To succeed it had to reach outside itself, use the right tool, recognize what had happened and know when to stop. Those requirements tied learned language to the physical realities of files, machines, networks, electricity and other people’s decisions. As their use expanded, repeated model calls added a new source of demand for accelerators, memory, networks and electricity. The next chapter turns from what agents could attempt to the infrastructure that decided how many of them could run, how fast, at what cost and within what physical limits.