Human-in-the-loop

Human-in-the-loop (HITL) describes any arrangement in which a person is a required participant in an automated system's operating cycle rather than a bystander to it. The phrase is used by two communities that rarely cite each other. In machine learning, it names the practice of drawing training signal from people: labeling data, answering queries the model chooses to ask, and ranking candidate outputs ^[4][5]. In safety policy and regulation, it names a control on what a deployed system may do on its own, usually by requiring a human decision before an action takes effect ^[1][21].

The two senses pull in opposite directions. As a training method, the human is a cost to be minimized: the whole point of active learning and preference learning is to buy the most model improvement per hour of human labor ^[10][12]. As a safeguard, the human is a cost to be protected: an overseer who is rushed, poorly informed, or facing hundreds of approvals a shift is not really overseeing anything, and a growing empirical literature argues that oversight duties are routinely assigned to people who cannot discharge them ^[34][35][36].

The vocabulary most often used to grade how much control a person retains comes from autonomous weapons policy. A 2012 Human Rights Watch report set out three positions for a human relative to a machine's decision: human-in-the-loop, human-on-the-loop, and human-out-of-the-loop ^[1]. Those labels are now applied to any system where the question is whether a person must approve an action, may override it after the fact, or is absent from it entirely.

Where the human sits in the loop

Scroll sideways for more →

US Department of Defense Directive 3000.09, reissued on 25 January 2023 in place of the November 2012 original, uses definitions that map onto the first and third positions without using the loop vocabulary in its policy text. A semi-autonomous weapon system is one that, once activated, "is intended to only engage individual targets or specific target groups that have been selected by an operator"; an autonomous weapon system is one that, once activated, "can select and engage targets without further intervention by an operator" ^[2][28]. The directive's central requirement is that both classes "be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force," a formulation that deliberately avoids specifying manual control ^[2][28].

The loop labels describe where a person sits, not how much they understand or can actually do. The main attempt to close that gap is the concept of meaningful human control, developed by Richard Moyes of the disarmament group Article 36 and Heather Roff, then a research scientist in Arizona State University's Global Security Initiative and a senior research fellow at the University of Oxford. Their 2016 briefing for delegates at the UN Convention on Certain Conventional Weapons lists four key elements: predictable, reliable and transparent technology; accurate information for the user about the intended outcome, the operation of the technology, and the context of use; timely human action and a potential for timely intervention; and accountability to a certain standard ^[3]. The same four elements read equally well as a specification for reviewing a loan decision or a content takedown.

Humans in the training loop

Data labeling and annotation

Supervised machine learning consumes labeled examples, and for most of the field's history those labels came from people. ImageNet, the dataset behind the 2012 deep learning breakthrough in image classification, was built by putting candidate images in front of Amazon Mechanical Turk workers who verified whether each image belonged to a given category ^[6]. The organizers' retrospective on five years of the ImageNet Large Scale Visual Recognition Challenge is blunt about how fragile that arrangement is: their headline lesson was that "All human intelligence tasks need to be exceptionally well-designed," because any ambiguity in a question was found by workers and degraded accuracy ^[6]. They also noted that the scale of the task, 1,000 object classes and more than a million images, made annotating the locations of all objects infeasible ^[6].

The data labeling industry that grew out of this is now large enough to be a strategic asset. Scale AI confirmed in June 2025 that Meta had invested roughly $14.3 billion for a 49% stake at a $29 billion valuation, that co-founder and chief executive Alexandr Wang was leaving to work on Meta's AI effort, and that chief strategy officer Jason Droege would take over as interim chief executive ^[7]. Mercor, which matches domain experts to AI labs producing training and evaluation data, raised a $350 million Series C at a $10 billion valuation in October 2025, five times its February 2025 valuation of $2 billion ^[8].

The labor conditions behind those numbers have been contested. A January 2023 TIME investigation reported that OpenAI used the outsourcing firm Sama to have Kenyan workers label text describing violence, sexual abuse and hate speech so that a safety filter could be trained for ChatGPT; workers were paid between roughly $1.32 and $2 per hour after taxes, while OpenAI paid Sama about $12.50 per hour, and Sama canceled the work in February 2022, eight months early ^[9].

Active learning

Active learning is the branch of HITL machine learning in which the model chooses what a person is asked to label. Burr Settles's survey, issued as University of Wisconsin-Madison Computer Sciences Technical Report 1648, states the premise directly: "a machine learning algorithm can achieve greater accuracy with fewer training labels if it is allowed to choose the data from which it learns," posing queries in the form of unlabeled instances to be labeled by an oracle, typically a human annotator ^[10]. The survey organizes the field by the scenario in which queries are formed (membership query synthesis, stream-based selective sampling, and pool-based sampling) and by the strategy used to pick a query, including uncertainty sampling, query-by-committee, expected model change, expected error reduction, variance reduction, and density-weighted methods ^[10]. The motivation is economic: unlabeled data is often abundant while labels are slow and costly ^[10].

Interactive machine learning and machine teaching

A separate tradition treats the human less as an oracle and more as a collaborator. Jerry Alan Fails and Dan R. Olsen Jr's 2003 paper at the Intelligent User Interfaces conference proposed an interactive machine learning model in which users train a classifier, view and correct its classifications, and iterate; they demonstrated it with Image Processing with Crayons, a tool for creating camera-based interfaces using a simple painting metaphor ^[11]. The survey by Mosqueira-Rey and colleagues in Artificial Intelligence Review distinguishes three paradigms by who is directing the process: active learning, where the system remains in control of what is asked; interactive machine learning, with tighter user and system interaction; and machine teaching, where a domain expert drives the process ^[5]. A 2022 survey by Wu and colleagues frames the same territory around cost, describing HITL as training an accurate model at minimum cost by folding human knowledge and experience into the pipeline ^[4].

Learning from human preferences

The most consequential HITL technique in current large language model practice is RLHF, reinforcement learning from human feedback, in which people compare candidate outputs and a reward model is fitted to those comparisons.

Scroll sideways for more →

Direct preference optimization simplified the machinery but not the dependency: it still needs pairs of outputs that a person has ranked ^[16]. The more direct attempt to reduce the human share is Constitutional AI, published by Anthropic in December 2022, which trains a harmless assistant without human labels identifying harmful outputs. In that method the only human oversight is a written list of rules or principles, and the preference model is trained on AI-generated comparisons, an approach the authors call reinforcement learning from AI feedback ^[17]. This is HITL moved up a level: humans still write the objective, but they no longer adjudicate individual cases.

Scalable oversight is the name for the limit case. It was set out as one of five concrete accident-risk problems by Amodei and colleagues in 2016: how to train a system toward an objective that is too expensive to evaluate frequently, when the true signal (what a careful human would say after hours of review) can only be sampled rarely ^[18]. Techniques associated with it, including debate, recursive reward modeling, and LLM-as-a-judge evaluation, all try to stretch a limited human budget over a larger volume of model behavior.

Oversight at deployment time

Deployment-side HITL shows up as approval gates, escalation paths, and review queues. The Model Context Protocol specification, a widely adopted interface for giving models access to external tools, writes the requirement into the standard: "For trust & safety and security, there SHOULD always be a human in the loop with the ability to deny tool invocations," with applications advised to show which tools are exposed, indicate visually when a tool runs, and present confirmation prompts ^[19]. The same document tells clients to prompt for confirmation on sensitive operations and to show tool inputs to the user before the call is made ^[19]. This is the pattern that AI agents generally follow.

Systems that must act in real time tend toward the on-the-loop position, because a per-action approval step is too slow. Waymo describes a "fleet response" function in which remote agents view real-time feeds from a vehicle's exterior cameras and a 3D graphical representation of what the car perceives, then answer questions put to them by the vehicle about ambiguous scenes, such as a construction site with an atypical cone configuration. The company states that "The Waymo Driver does not rely solely on the inputs it receives from the fleet response agent and it is in control of the vehicle at all times," which places the human outside the real-time control loop and inside an advisory one ^[20].

Regulation and legal requirements

Human oversight is one of the few AI safety practices written into binding law. Article 14 of the EU AI Act (Regulation (EU) 2024/1689) requires that high-risk AI systems be designed so that they "can be effectively overseen by natural persons during the period in which they are in use," with the oversight measures commensurate with the risks, level of autonomy and context of use of the system ^[21]. Article 14(4) enumerates what the overseer must be able to do: understand the system's capacities and limitations, remain aware of automation bias, interpret the output correctly, decide not to use the system or to override its output, and intervene or interrupt through a stop function ^[21]. Article 14(5) adds a two-person rule for remote biometric identification systems: no action may be taken on an identification unless it "has been separately verified and confirmed by at least two natural persons," subject to a carve-out where Union or national law considers this disproportionate for law enforcement, migration, border control and asylum uses ^[21]. Article 26(2) puts the matching duty on deployers, who "shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support" ^[22]. Article 4, applicable since 2 February 2025, requires providers and deployers to ensure a sufficient level of AI literacy among staff operating their systems ^[23].

The comparable provision in data protection law predates the AI Act. Article 22 of the GDPR restricts decisions based solely on automated processing that produce legal or similarly significant effects, and where such processing is permitted the controller must safeguard the data subject's rights including, at minimum, "the right to obtain human intervention on the part of the controller" ^[27].

The AI Act's own timetable moved in 2026. Under Article 113 the regulation applies in stages, with prohibitions from 2 February 2025, general-purpose AI and governance rules from 2 August 2025, and the bulk of the regulation from 2 August 2026 ^[24]. A simplification package known as the digital omnibus pushed the high-risk obligations, Article 14 among them, back further: the European Parliament approved it on 16 June 2026 by 423 votes to 57 with 174 abstentions, moving compliance for standalone Annex III high-risk systems to 2 December 2027 and for AI embedded in regulated products to 2 August 2028, while adding a prohibition on systems generating non-consensual intimate imagery or child sexual abuse material ^[25]. The Council gave final approval on 29 June 2026 ^[26].

Autonomous weapons and lethal decisions

The sharpest version of the debate is about whether a machine may select and apply lethal force without a person deciding. The International Committee of the Red Cross took a formal position in May 2021 recommending that unpredictable autonomous weapon systems be ruled out, that systems designed to apply force against persons be prohibited, and that others be regulated through limits on targets, duration, geographic scope and scale of use, together with requirements for human-machine interaction that ensure effective human supervision and timely intervention and deactivation ^[29].

On 5 October 2023, UN Secretary-General António Guterres and ICRC President Mirjana Spoljaric issued a joint call for states to negotiate a legally binding instrument on autonomous weapons and to conclude those negotiations by 2026 ^[30]. The General Assembly adopted a resolution on lethal autonomous weapons systems on 2 December 2024 by 166 votes to 3, with Belarus, North Korea and Russia opposed and 15 states abstaining ^[32], and on 6 November 2025 its First Committee adopted a further resolution on autonomous weapons systems by 156 votes to 5 with 8 abstentions, calling on parties to the Convention on Certain Conventional Weapons to complete the set of elements being developed by the Group of Governmental Experts with a view to future negotiation ^[31]. That formulation still treats negotiation of a binding text as a future step rather than one under way ^[31].

The case most often cited against assuming that a supervising operator supplies real control is the US Patriot air and missile defense system during the 2003 invasion of Iraq. In a 2017 report for the Center for a New American Security, John K. Hawley documented that two of the 11 ballistic missile engagements Patriot units conducted during that campaign were fratricides, destroying a Royal Air Force Tornado and a US Navy F/A-18 ^[33]. Hawley attributes the failures to a combination of machine error and the human performance problems of supervisory control over a largely automated system, including long stretches of monotonous monitoring punctuated by seconds of decision, few practical points at which crews could intervene, and training and doctrine that encouraged operators to trust the system's classifications ^[33].

Does human oversight work?

The empirical record on whether people can perform the oversight that policies assume is not encouraging. Raja Parasuraman and Dietrich Manzey's 2010 review in Human Factors integrated two previously separate literatures, automation-related complacency in supervisory control and automation bias in decision support, arguing that both arise from how attention is allocated when a reliable-seeming aid is present ^[34]. The EU AI Act's drafters were aware of this: Article 14(4)(b) names automation bias explicitly as something the design must help overseers resist ^[21].

Ben Green and Yiling Chen tested the assumption directly. In a controlled experiment on Amazon Mechanical Turk covering pretrial release and financial lending, participants making predictions with the aid of a risk assessment "were unable to effectively evaluate the accuracy of their own or the risk assessment's predictions," "did not calibrate their reliance on the risk assessment based on the risk assessment's performance," and "exhibited bias in their interactions with the risk assessment," under every presentation condition tested ^[35]. Green extended the argument to policy in a 2022 article in Computer Law & Security Review, surveying 41 policies that prescribe human oversight of government algorithms and finding two flaws: people cannot perform the oversight functions the policies describe, and as a result the policies legitimize the use of faulty and contested tools without addressing what is wrong with them ^[36].

Madeleine Clare Elish's 2019 paper in Engaging Science, Technology, and Society names the accountability consequence a "moral crumple zone," in which responsibility for a failure lands on the human operator nearest the machine even though that person had limited control over its behavior ^[37]. The clearest illustration is the National Transportation Safety Board's investigation of the March 2018 Uber Advanced Technologies Group crash in Tempe, Arizona, which killed a pedestrian. The Board found the probable cause to be the vehicle operator's failure to monitor the driving environment and the operation of the automated driving system because she was visually distracted throughout the trip by her personal cell phone. Contributing factors included the company's inadequate safety risk assessment procedures, its ineffective oversight of vehicle operators and its lack of adequate mechanisms for addressing operators' automation complacency, together with the pedestrian's crossing outside a crosswalk and Arizona's insufficient oversight of automated vehicle testing ^[38].

Developments in 2025 and 2026

The statutory deadline that would have made Article 14 operative slipped by more than a year, from 2 August 2026 to 2 December 2027 for standalone high-risk systems and 2 August 2028 for AI embedded in regulated products ^[25][26]. That leaves the sector's own conventions doing most of the work in the interim.

On the deployment side, those conventions are increasingly written into protocols rather than left to individual products. The Model Context Protocol places the confirmation requirement on the client application, not on the user: the specification says the client should surface which tools are exposed, signal when one runs, and prompt before sensitive operations ^[19]. That design choice matters for agentic AI, where a single task can produce many tool calls and the reviewer sees only what the client chooses to show.

On the training side, the human share of the loop is shrinking in some places and becoming more specialized in others. Constitutional AI and related methods substitute model-generated comparisons for human ones, leaving humans to write the governing principles rather than to judge individual outputs ^[17], and synthetic data covers ground that once required annotation. At the same time the money has moved toward expert human work: Mercor's October 2025 round valued a business built on paying domain experts for training and evaluation data at $10 billion ^[8].

See also

• RLHF
• Active learning
• Data labeling
• Autonomous weapons
• AI safety
• Automation bias

References

1. ^Human Rights Watch and Harvard Law School International Human Rights Clinic, "Losing Humanity: The Case against Killer Robots," 19 November 2012. hrw.org/...case-against-killer-robots
2. ^Congressional Research Service, "Defense Primer: U.S. Policy on Lethal Autonomous Weapon Systems" (IF11150), updated 26 March 2026. everycrsreport.com/...IF11150
3. ^Heather M. Roff and Richard Moyes, "Meaningful Human Control, Artificial Intelligence and Autonomous Weapons," briefing paper for the CCW Informal Meeting of Experts on Lethal Autonomous Weapons Systems, Geneva, 11-15 April 2016 (Article 36 and Arizona State University). article36.org/...MHC-AI-and-AWS-FINAL.pdf
4. ^Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma and Liang He, "A Survey of Human-in-the-loop for Machine Learning," arXiv:2108.00941 (2021), published in Future Generation Computer Systems, 2022. arxiv.org/...2108.00941
5. ^Eduardo Mosqueira-Rey et al., "Human-in-the-loop machine learning: a state of the art," Artificial Intelligence Review 56(4), 2023, pp. 3005-3054. link.springer.com/...s10462-022-10246-w
6. ^Olga Russakovsky et al., "ImageNet Large Scale Visual Recognition Challenge," arXiv:1409.0575 (2014). arxiv.org/...1409.0575
7. ^Ivan Mehta, "Scale AI confirms 'significant' investment from Meta, says CEO Alexandr Wang is leaving," TechCrunch, 13 June 2025. techcrunch.com/...ays-ceo-alexandr-wang-is-leaving
8. ^TechCrunch, "Mercor quintuples valuation to $10B with $350M Series C," 27 October 2025. techcrunch.com/...uation-to-10b-with-350m-series-c
9. ^Billy Perrigo, "Exclusive: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic," TIME, 18 January 2023. time.com/...openai-chatgpt-kenya-workers
10. ^Burr Settles, "Active Learning Literature Survey," Computer Sciences Technical Report 1648, University of Wisconsin-Madison, updated 26 January 2010. burrsettles.com/...settles.activelearning.pdf
11. ^Jerry Alan Fails and Dan R. Olsen Jr, "Interactive Machine Learning," Proceedings of the 8th International Conference on Intelligent User Interfaces (IUI '03), 2003. dl.acm.org/...604045.604056
12. ^Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg and Dario Amodei, "Deep reinforcement learning from human preferences," arXiv:1706.03741, 12 June 2017. arxiv.org/...1706.03741
13. ^Nisan Stiennon et al., "Learning to summarize from human feedback," arXiv:2009.01325, 2 September 2020. arxiv.org/...2009.01325
14. ^Long Ouyang et al., "Training language models to follow instructions with human feedback," arXiv:2203.02155, 4 March 2022. arxiv.org/...2203.02155
15. ^Yuntao Bai et al., "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback," arXiv:2204.05862, 12 April 2022. arxiv.org/...2204.05862
16. ^Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning and Chelsea Finn, "Direct Preference Optimization: Your Language Model is Secretly a Reward Model," arXiv:2305.18290, 29 May 2023. arxiv.org/...2305.18290
17. ^Yuntao Bai et al., "Constitutional AI: Harmlessness from AI Feedback," arXiv:2212.08073, 15 December 2022. arxiv.org/...2212.08073
18. ^Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman and Dan Mané, "Concrete Problems in AI Safety," arXiv:1606.06565, 21 June 2016. arxiv.org/...1606.06565
19. ^Model Context Protocol specification, version 2025-06-18, "Tools." modelcontextprotocol.io/...tools
20. ^Waymo, "Fleet response: Lending a helpful hand to Waymo's autonomously driven vehicles," 21 May 2024. waymo.com/...fleet-response
21. ^EU AI Act, Regulation (EU) 2024/1689, Article 14 (Human oversight). artificialintelligenceact.eu/...14
22. ^EU AI Act, Article 26 (Obligations of deployers of high-risk AI systems). artificialintelligenceact.eu/...26
23. ^EU AI Act, Article 4 (AI literacy). artificialintelligenceact.eu/article
24. ^EU AI Act, Article 113 (Entry into force and application). artificialintelligenceact.eu/...113
25. ^European Parliament, "AI Act: EP approves simplification measures and 'nudifier' app ban," press release, 16 June 2026. europarl.europa.eu/...easures-and-nudifier-app-ban
26. ^Council of the EU, "Artificial intelligence: Council gives final green light to simplify and streamline rules," press release, 29 June 2026. consilium.europa.eu/...mplify-and-streamline-rules
27. ^General Data Protection Regulation, Article 22 (Automated individual decision-making, including profiling). gdpr-info.eu/art-22-gdpr
28. ^US Department of Defense Directive 3000.09, "Autonomy in Weapon Systems," 25 January 2023. esd.whs.mil/...300009p.pdf
29. ^International Committee of the Red Cross, "ICRC position on autonomous weapon systems," 12 May 2021. icrc.org/...icrc-position-autonomous-weapon-systems
30. ^ICRC and United Nations, "Joint call by the United Nations Secretary-General and the President of the International Committee of the Red Cross for States to establish new prohibitions and restrictions on Autonomous Weapon Systems," 5 October 2023. icrc.org/...estrictions-autonomous-weapons-systems
31. ^Stop Killer Robots, "156 states support UNGA resolution," 6 November 2025. stopkillerrobots.org/...es-support-unga-resolution
32. ^Human Rights Watch, "UN: Start Talks on Treaty to Ban 'Killer Robots'," 21 May 2025 (reporting the General Assembly resolution on lethal autonomous weapons systems adopted 2 December 2024 by 166-3-15). hrw.org/...un-start-talks-treaty-ban-killer-robots
33. ^John K. Hawley, "Patriot Wars: Automation and the Patriot Air and Missile Defense System," Center for a New American Security, 25 January 2017. cnas.org/...patriot-wars
34. ^Raja Parasuraman and Dietrich H. Manzey, "Complacency and Bias in Human Use of Automation: An Attentional Integration," Human Factors 52(3), June 2010. doi.org/...0018720810376055
35. ^Ben Green and Yiling Chen, "The Principles and Limits of Algorithm-in-the-Loop Decision Making," Proceedings of the ACM on Human-Computer Interaction 3, CSCW, 2019. doi.org/...3359152
36. ^Ben Green, "The Flaws of Policies Requiring Human Oversight of Government Algorithms," Computer Law & Security Review 45, 2022; preprint arXiv:2109.05067. arxiv.org/...2109.05067
37. ^Madeleine Clare Elish, "Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction," Engaging Science, Technology, and Society 5, 2019, pp. 40-60. doi.org/...ests2019.260
38. ^National Transportation Safety Board, investigation HWY18MH010, "Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018," report HAR-19-03. ntsb.gov/...HWY18MH010