# Imitation Learning

> Source: https://aiwiki.ai/wiki/imitation_learning
> Updated: 2026-07-29
> Fact-checked: 2026-07-29
> Categories: Machine Learning, Reinforcement Learning, Robotics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Imitation Learning." aiwiki.ai, 29 Jul 2026. https://aiwiki.ai/wiki/imitation_learning
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**Imitation learning** is a family of methods for learning sequential behavior from demonstrations. A demonstrator provides examples of what it did, usually as trajectories of observations or states paired with actions, and a learner uses those examples to construct a policy. The term overlaps with *learning from demonstration* and *apprenticeship learning*. Demonstrations may come from a person, another robot, a controller, or a planner. Some methods copy the demonstrated action choices directly, while others infer a reward or match the distribution of demonstrated behavior.[1][2]

Imitation learning is used when suitable behavior is easier to demonstrate than to specify with a complete reward function. It is closely related to [supervised learning](https://aiwiki.ai/wiki/supervised_learning), [reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning), optimal control, and [robotics](https://aiwiki.ai/wiki/robotics), but it is not identical to any one of them. A supervised imitation policy changes the future inputs it will receive by acting in the world, and this feedback can make small prediction errors compound. Interactive data collection, reward inference, occupancy-measure matching, action-sequence models, and methods that learn from observation-only video address different parts of that problem.[1][8][9]

The field spans several settings rather than a single algorithm. Behavior cloning treats demonstrated state-action pairs as labeled examples. Interactive methods such as DAgger ask an expert to label states visited by the learner. Inverse reinforcement learning seeks a reward that explains the demonstrations. Adversarial methods compare learner and expert trajectories. Modern visuomotor policies can predict action chunks with transformers or conditional diffusion models. These approaches require different data and make different assumptions, so results obtained by one should not be treated as guarantees for the others.[1][9][10][15][16]

## Problem formulation

A common formalization uses a [Markov decision process](https://aiwiki.ai/wiki/markov_decision_process_mdp) with state space *S*, action space *A*, transition law *P*, initial-state distribution, horizon or discount factor, and a policy that selects actions. In ordinary reinforcement learning, a reward function defines the return to optimize. In imitation learning, the learner instead receives a demonstration dataset. A trajectory can be written as a sequence `(s0, a0, s1, a1, ..., sT)`, although practical datasets often contain images or other observations rather than the complete state.[1][2]

The demonstrations do not have to be generated by a human. Argall and colleagues' robotics survey distinguishes demonstrations produced through teleoperation, kinesthetic teaching, sensors placed on a teacher, and external observation. A controller or planner may also serve as the demonstrator. The word *expert* usually denotes the policy that generated the target examples; it does not prove that the demonstrator is optimal, consistent, or safe.[2]

In the simplest behavior-cloning formulation, the dataset contains pairs `(o, a)` sampled from expert trajectories. A parameterized policy is trained to minimize an empirical prediction loss:

`L_BC(theta) = average l(pi_theta(o), a)`

The loss may be cross-entropy for discrete actions, squared error or another regression loss for continuous actions, or the negative log likelihood of an action under a probabilistic policy. This objective measures agreement on observations represented in the training data. It does not by itself measure the quality of full rollouts from the learned policy.[1][8]

Other formulations operate on distributions of trajectories or state-action visits. Inverse reinforcement learning searches for a reward under which the demonstrated behavior is high-valued or approximately optimal. Occupancy-matching methods seek a learner whose discounted state-action visitation distribution resembles the demonstrator's. Observation-only methods may compare state or visual observation distributions, learn an inverse-dynamics model to infer missing actions, or derive a reward from progress in a learned representation.[5][7][10][24][25]

The available information matters. Some datasets contain true states and actions; others contain camera images, proprioception, language instructions, timestamps, or only video. The learner may know the transition model, have access to a simulator, be allowed to execute exploratory rollouts, or be restricted to a fixed offline dataset. These are materially different problems. For example, DAgger requires new rollouts and an expert that can label learner-visited states, while conventional offline behavior cloning requires neither. GAIL does not query the expert after the original demonstrations, but its policy-training loop still interacts with an environment.[9][10]

Partial observability can also change what is learnable. If the policy sees only an image that omits a relevant latent variable, two apparently identical observations may call for different actions. A history-dependent model can sometimes infer hidden state from prior observations, but no architecture can recover information absent from the observations and their history. Demonstration data can also contain temporally correlated disturbances that create misleading correlations between recorded states and actions.[13]

## Relationship to supervised and reinforcement learning

Behavior cloning uses supervised-learning tools, but its deployment is sequential. In a conventional independent and identically distributed prediction problem, a model's output does not ordinarily determine the distribution of its next input. In control, a steering, locomotion, or manipulation error changes the next state. Ross and Bagnell formalized how this mismatch can yield error that grows quadratically with the horizon in a worst-case finite-horizon analysis of naive supervised imitation.[8]

Reinforcement learning and imitation learning differ mainly in their training signals. Reinforcement learning evaluates behavior through reward and can improve through environment interaction. Imitation learning uses demonstrated behavior and may require no task reward. Hybrid systems can first imitate a dataset and then optimize a reward, use demonstrations to shape exploration, or learn a reward from demonstrations before performing reinforcement learning. Consequently, "imitation learning" does not imply that the final system never uses reward or exploration.[1][6][30]

| Setting | Primary training signal | New environment interaction | Expert access after the initial dataset | Typical learned object |
|---|---|---:|---:|---|
| Offline behavior cloning | Demonstrated observation-action pairs | No | No | Policy |
| Interactive imitation | Demonstrations plus labels on learner-visited states | Yes | Usually yes | Policy |
| Inverse reinforcement learning | Demonstrated trajectories | Often needed for planning or policy optimization | Not necessarily | Reward or cost, then policy |
| Adversarial occupancy matching | Expert trajectories plus learner rollouts | Yes | No | Policy and discriminator |
| Reinforcement learning | Scalar reward obtained from interaction or a fixed dataset | Depends on online or offline setting | No | Value function and/or policy |

The table describes information flow, not a ranking. A fixed set of high-quality demonstrations can make behavior cloning practical where online exploration would be unsafe or expensive. Conversely, a learned policy may need reward-based improvement when the demonstrations are incomplete or suboptimal. Inverse reinforcement learning can support re-optimization under changed circumstances, but recovered rewards are not uniquely determined by behavior without assumptions or additional information.[5][11]

## Demonstrations and data collection

### Demonstrator and interface

Demonstrations can be collected by recording a person who directly controls the target system, by physically guiding a robot through a motion, by observing a separate teacher, or by executing a hand-designed controller. Teleoperation records the target robot's own observations and actions and therefore avoids some cross-embodiment mapping problems. Kinesthetic teaching can be intuitive for manipulators but is limited by the hardware that can be safely moved by hand. External video is abundant, yet usually lacks the action labels and robot-centric observations required by direct behavior cloning.[2]

The data-collection interface affects the distribution and meaning of the labels. A joystick may command end-effector velocity, while joint-space teleoperation records motor targets and a planner may label a high-level waypoint. Latency, controller smoothing, camera placement, and intervention rules can change the learned relationship between observations and actions. A publication should therefore report what an action represents, how observations and commands were synchronized, and whether a lower-level controller executed the recorded command.[2][15]

### Correspondence and embodiment

Learning from a different body introduces a *correspondence problem*. Argall and colleagues separate mapping the recorded teacher signal into a representation usable by the learner from mapping behavior across different embodiments. A human hand, a gripper, and a mobile manipulator do not share the same joints or action space. Even when both agents perform the same task, pixel-level appearance and feasible motion can differ substantially.[2]

Cross-embodiment methods avoid demanding exact action correspondence. XIRL, for example, learns a visual embedding of task progress from videos made by agents with different embodiments, then uses distance to the goal in that embedding as a reward for reinforcement learning. This is an empirical approach to learning a transferable progress signal, not a general proof that videos from any body can teach any other body.[25]

### Action-free and imperfect data

Observation-only learning replaces the expert's action labels with additional assumptions or models. Sun and colleagues formalized imitation learning from observation alone and proposed matching observation distributions in a time-dependent policy. Video PreTraining, applied to Minecraft, used a smaller action-labeled dataset to train an inverse-dynamics model, applied that model to unlabeled online videos, and then behavior-cloned the inferred actions. Both examples show that action-free data can be useful, but neither makes action labels unnecessary in every setting.[24][26]

Demonstrations may contain pauses, corrections, failed attempts, or systematic bias. Simply mixing trajectories of unequal quality treats all recorded actions as equally desirable. Methods for imperfect demonstrations instead select useful transitions, learn rankings or preferences, infer rewards, or condition policies on a measure of performance. D-REX generated ranked trajectories by adding increasing noise to a behavior-cloned policy, learned a reward from those rankings, and optimized the reward with reinforcement learning on the tested simulated robot and Atari domains.[30][31]

Data quantity is not a substitute for coverage. Repeated demonstrations of one nominal trajectory may leave recovery states unrepresented. Conversely, broader data can make a cloned policy less brittle if it includes the variations that deployment will encounter. The relevant variables include initial conditions, object configurations, demonstrator styles, sensor conditions, and failure recoveries, not only the number of episodes.[8][12]

## Main method families

### Behavior cloning

Behavior cloning directly learns the expert's conditional action distribution. It is attractive because optimization is supervised, training examples can be shuffled and batched, and the environment is not needed during training. The policy can be a linear model, a [neural network](https://aiwiki.ai/wiki/neural_network), a recurrent model, a [transformer](https://aiwiki.ai/wiki/transformers), or another conditional density estimator. At deployment, the policy maps current observations, and sometimes a history or instruction, to an action or action sequence.[1]

For a deterministic continuous controller, minimizing mean squared error estimates a conditional mean. This can be unsuitable when several distinct actions are valid. If demonstrations pass around an obstacle on both the left and the right, averaging the two action modes could point toward the obstacle. Mixture models, latent-variable policies, autoregressive decoders, energy-based policies, and diffusion models are ways to represent multimodal action distributions. Their ability to represent alternatives does not ensure that the chosen mode remains consistent across a long trajectory.[1][16]

Behavior cloning is often called an offline method because it can train from a fixed dataset. That property should be separated from deployment safety. A low validation loss on held-out expert pairs demonstrates interpolation on the expert data distribution; it does not establish successful closed-loop behavior. Rollout evaluation remains necessary because policy errors alter future observations.[8][9]

### Interactive imitation and DAgger

Interactive imitation collects supervision on states that the learner actually visits. DAgger, introduced by Ross, Gordon, and Bagnell, alternates between rolling out a policy, asking the expert what action it would take on visited states, adding those labeled states to an aggregate dataset, and training the next policy on that aggregate. Mixtures with the expert can be used during early iterations so that an untrained learner does not immediately enter arbitrary states.[9]

The theoretical contribution of DAgger is a reduction to no-regret online learning. Under the paper's assumptions, a no-regret learner can produce a policy with low surrogate loss under the state distribution induced by the learned policy, rather than only under the expert's distribution. The guarantee is conditional on the quality and availability of expert labels and on the relationship between the surrogate loss and task cost.[9]

Human-in-the-loop use creates practical burdens. Labeling every learner-visited state can be tiring, expensive, or unsafe, and the expert may find isolated counterfactual labels difficult to provide. DART addresses a different point in the design space: it injects calibrated noise while the supervisor demonstrates so that the offline dataset includes recovery actions. In the paper's simulation and Toyota HSR grasping experiments, DART improved on ordinary behavior cloning, but those results do not eliminate the need to evaluate the method for a new task or safety regime.[12]

Intervention-based systems can also let a human take control only when the policy approaches failure. Such data emphasizes difficult states, but it introduces choices about when to intervene, how to label the transition into and out of an intervention, and how to avoid a dataset dominated by nominal autonomous behavior. Those choices are part of the method, not merely an implementation detail.[2]

### Inverse reinforcement learning

Inverse reinforcement learning asks what reward or cost could explain demonstrated behavior. Ng and Russell formulated algorithms for finding reward functions under which an observed policy is optimal. The problem is underdetermined: constant rewards and many other reward functions can make the same policy optimal. IRL therefore relies on assumptions, constraints, priors, or a probabilistic model of the demonstrator.[5]

Abbeel and Ng's apprenticeship-learning method assumes that reward is a linear combination of known features. It iteratively finds policies whose expected feature counts approach the expert's feature expectations. Under that model, matching feature expectations can yield a policy close to the expert in return even without identifying the expert's exact reward weights.[6]

Maximum-entropy IRL defines a probability distribution over trajectories that favors high-reward behavior while retaining maximum entropy subject to feature constraints. Ziebart and colleagues developed this approach for noisy and imperfect route-choice data, including 100,000 miles of taxi GPS trajectories. The probabilistic formulation addresses ambiguity over behavior consistent with the constraints, but the choice of features, dynamics model, and rationality assumptions still matters.[7]

A learned reward can be optimized in new initial states or dynamics, which is one reason to infer an objective instead of only copying actions. That transfer is not automatic. Reward shaping can produce behaviorally equivalent rewards under one dynamics model that behave differently after the dynamics change. AIRL introduced an adversarial reward-learning structure intended to separate a transferable state-based reward from dynamics-dependent shaping under stated assumptions, and reported stronger transfer in its experiments.[11]

### Occupancy matching and adversarial imitation

GAIL connects imitation learning to [generative adversarial networks](https://aiwiki.ai/wiki/generative_adversarial_network). A discriminator learns to distinguish expert state-action samples from learner samples. The learner's policy is optimized through reinforcement learning to make its visitation distribution difficult to distinguish from the expert's. This bypasses an explicit, separately recovered reward in the basic algorithm.[10]

GAIL is model-free in the sense that it does not require a known transition model, but it is not environment-free. It needs fresh policy rollouts and a reinforcement-learning optimization loop. Training can inherit the instability and sample demands of adversarial and policy-gradient optimization. In addition, matching the demonstrated occupancy distribution does not by itself show how a policy will behave in states that neither the expert nor the trained learner visited.[10]

AIRL uses a related adversarial procedure but structures the discriminator so that a reward can be recovered. The paper's transfer claims are tied to assumptions about the reward and dynamics and to its experimental settings. It is therefore more accurate to describe AIRL as a method designed for reward transfer than to state that adversarial IRL always recovers a true or causal reward.[11]

### Action chunks and sequence policies

Per-step policies make a new prediction at every control step. Sequence policies instead predict a block of future actions, often execute part of it, then replan from a new observation. This can represent temporal coordination and reduce the number of high-level prediction points. It also introduces design choices about chunk length, overlap, temporal ensembling, and how quickly the system can react to unexpected events.[15][16]

[Action Chunking with Transformers](https://aiwiki.ai/wiki/action_chunking_transformer), or ACT, is a conditional variational sequence model developed with the ALOHA bimanual teleoperation system. It predicts action chunks and uses temporal ensembling across overlapping predictions. In the paper, the system learned six real-world fine-manipulation tasks, and the authors reported 80 to 90 percent success on tasks such as opening a translucent condiment cup and inserting a battery from about ten minutes of demonstrations per task. These are results for the reported hardware, tasks, data, and evaluation protocol, not a general ten-minute learning guarantee.[15]

Diffusion Policy represents a conditional action-sequence distribution as a denoising [diffusion model](https://aiwiki.ai/wiki/diffusion_model). Its design combines visual conditioning, receding-horizon control, and iterative denoising in action space. The paper reports evaluation on 15 tasks from four robot-manipulation benchmarks and an average 46.9 percent improvement over the compared methods. The number is an aggregate within that benchmark suite and should not be read as a fixed advantage over every behavior-cloning architecture.[16]

Action chunks can improve smoothness and express multimodal behavior, but they also commit the system to predictions made from an earlier observation. Receding-horizon execution and temporal ensembling partly restore feedback. The appropriate balance depends on control frequency, sensing latency, task dynamics, and the consequence of delayed reaction.[15][16]

### Imitation from observations

When demonstrations lack action labels, direct state-action regression is unavailable. One strategy learns inverse dynamics from a smaller action-labeled dataset and predicts the missing actions between observed frames. Another learns a reward or representation that measures progress and then uses reinforcement learning. A third matches learner and expert observation distributions without reconstructing the expert's exact actions.[24][25][26]

Each strategy requires some bridge between the observed demonstrator and the learner. Inverse dynamics assumes that observed transitions provide enough information to infer actions relevant to the learner. Visual reward methods assume that task progress can be represented despite viewpoint and embodiment differences. Distribution matching requires an environment in which the learner can generate comparison trajectories. These assumptions should be evaluated explicitly when using web video, human demonstrations, or another robot's data.[24][25]

Video PreTraining illustrates a semi-supervised bridge. The authors collected a smaller Minecraft dataset with video and keyboard-mouse actions, trained an inverse-dynamics model, used it to label a much larger collection of online videos, and behavior-cloned those inferred labels. The resulting prior was then fine-tuned with imitation and reinforcement learning. This is a pipeline combining labeled data, unlabeled video, imitation, and reward optimization, rather than imitation from raw video alone.[26]

## Error compounding and other limitations

### Covariate shift

The central behavior-cloning problem is covariate shift between expert and learner trajectories. Suppose a supervised policy makes an error with probability *epsilon* on states sampled from the expert and the task lasts *T* steps. In the worst-case analysis of Ross and Bagnell, a first mistake can lead to unfamiliar states for the remaining horizon, yielding expected cost or regret proportional to `T^2 epsilon`. This is a bound under a particular finite-horizon setup, not a prediction that every policy's error will grow at exactly that rate.[8]

DAgger changes the training distribution by labeling learner-visited states. Under its reduction assumptions, the policy selected from the online sequence can achieve loss controlled under its own induced distribution, leading to a horizon dependence closer to linear in the relevant error term. That result does not guarantee safe exploration, correct expert labels, or recovery from states for which the expert itself has no suitable action.[9]

The classic discrete-action analysis is not the last word for continuous control. Simchowitz, Pfrommer, and Jadbabaie construct stable continuous state-action systems with smooth deterministic experts in which every smooth deterministic imitator can have execution error exponentially larger, as a function of horizon, than its error on the expert distribution. Their result is an existence theorem with specific conditions. It shows that stability of the dynamics alone is not sufficient for a universal behavior-cloning guarantee and motivates attention to policy class, stochasticity, history dependence, action chunking, and the spread of the expert trajectory distribution.[14]

### Demonstrator quality and ambiguity

An imitation policy inherits information from its demonstrations, including undesirable regularities. A demonstrator may be suboptimal, may use hidden information unavailable to the learner, or may take different actions in situations that look identical to the learner. Data aggregation cannot resolve inconsistent labels caused by missing observations. A probabilistic or history-dependent policy can represent uncertainty, but uncertainty representation is not the same as recovering the missing cause.[2][13]

Temporally correlated noise deserves particular care. Swamy and colleagues analyze settings in which persistent disturbances in expert actions create spurious correlations between later observed states and actions. A learner may copy a corrective pattern as though the visible state caused it, even when both were consequences of an unobserved disturbance. Their proposed instrumental-variable methods target that setting, but the broader lesson is that recorded correlation need not identify the expert's policy under confounding.[13]

Multimodality creates another ambiguity. If multiple action sequences solve a task, a single deterministic regression target can average incompatible strategies. Conditional generative policies can represent several modes, but the training data must still reveal which modes are valid and the controller must maintain temporal consistency after selecting one.[16]

### Reward ambiguity

Behavior alone does not identify a unique reward. Adding a constant, applying some forms of potential-based shaping, or changing the relative values of unvisited states can leave observed optimal behavior unchanged. Maximum-entropy models, feature constraints, priors, comparisons, and environmental changes narrow the set of explanations but do not make reward inference assumption-free.[5][7][11]

The distinction matters when a learned reward is reused outside the demonstration setting. Two rewards that explain the same training trajectories can favor different behavior under new dynamics or in unobserved states. Transfer evaluation should therefore test the learned reward after re-optimization in the intended environments, rather than only measuring how well its optimal policy reproduces the original demonstrations.[11]

### Safety and coverage

Demonstrations show a finite subset of possible behavior. They do not certify what the policy will do under sensor faults, novel obstacles, adversarial inputs, or rare combinations of familiar conditions. Interactive collection can add recoveries, and noise injection can expose the demonstrator to nearby off-trajectory states, but neither exhausts an open-ended environment.[9][12]

For safety-relevant deployment, imitation accuracy is only one component of evidence. Independent constraints, monitoring, fallback controllers, uncertainty tests, scenario coverage, and staged real-world evaluation may be needed. The exact measures depend on the system and risk. A benchmark success rate should not be translated into a safety claim unless the evaluation was designed to support that claim.

### Better-than-demonstrator performance

Ordinary behavior cloning minimizes disagreement with its labels and has no direct objective to exceed the demonstrator. A system can nevertheless appear better on a particular metric by averaging noisy actions, exploiting regularity across demonstrations, or combining imitation with planning or reward optimization. Such an outcome is empirical, not guaranteed by cloning.[1][30]

D-REX provides one explicit route beyond direct copying. It creates trajectories of ranked quality by adding varying noise to a cloned policy, infers a reward from the ranking, and optimizes that reward with reinforcement learning. The authors reported better-than-demonstrator results on selected simulated control and Atari benchmarks. This demonstrates a method under tested conditions, not a universal upper-bound removal for imitation learning.[30]

## Evaluation and reporting

### Closed-loop task performance

The most direct evaluation runs the learned policy in the target environment from a declared distribution of initial conditions and measures task success or return. Continuous metrics such as distance, completion time, collision count, intervention rate, constraint violations, and energy use can expose differences hidden by a binary success score. The metric should match the task claim and should not be selected after viewing test outcomes.

The number of trials matters, especially when failures are rare or success probabilities are near one another. Reports should give the number of independent trials and uncertainty intervals or the underlying counts. Repeating multiple episodes from nearly identical initial states does not establish robustness to wider deployment variation.

### Imitation metrics

Held-out negative log likelihood, classification error, or action regression error can diagnose whether the policy predicts expert labels. These metrics are useful for model selection, but they do not replace rollouts because they are measured on the demonstrated state distribution. A model with slightly lower action error can still fail more often if its errors occur at high-consequence states or lead to unrecoverable trajectories.[8][9]

Trajectory similarity can also be misleading. A robot may accomplish the task with a path different from the human's, while a visually similar trajectory may violate an unmeasured force or safety constraint. If exact style reproduction is a requirement, similarity should be measured alongside task and constraint metrics rather than assumed to imply them.

### Distribution shift and ablation

Generalization claims should name the shift being tested: new object instances, camera backgrounds, instructions, initial poses, dynamics, environments, or robot embodiments. Pooling unlike shifts into a single number can hide which capability improved. For a learned reward, re-optimization under changed dynamics is a stronger transfer test than executing the original policy unchanged.[11][17]

Ablations help identify what produced a result. For action-chunking or diffusion policies, relevant factors include chunk length, temporal ensembling, observation history, generative objective, data scale, and pretrained visual representation. For interactive imitation, report expert-query count and learner rollouts as well as final success. For large cross-embodiment datasets, compare in-domain data alone with the pooled dataset to measure transfer rather than attributing all improvement to scale.[9][15][16][19]

### Reproducibility boundaries

Robot-learning results are tied to hardware, controller frequency, calibration, teleoperation interface, scene construction, and success criteria. Papers can make these boundaries clearer by releasing code and data, documenting exclusions and reset procedures, and distinguishing simulation from physical trials. Even with released artifacts, a reported percentage is evidence about the stated protocol, not a universal property of the algorithm.

## Historical development

Imitation by machine predates the modern terminology, and it is risky to assign a single invention date. The following milestones document influential formulations and systems rather than an exhaustive origin story.

| Year | Work | Contribution and evidentiary scope |
|---:|---|---|
| 1988 | ALVINN | Pomerleau described a three-layer backpropagation network mapping camera and laser-range inputs to 45 steering-direction outputs. The reported network was trained on 1,200 simulated road snapshots and tested on a CMU outdoor path.[3] |
| 1990 | Rapidly Adapting Artificial Neural Networks for Autonomous Navigation | A later ALVINN system learned steering from a human driver's actions during on-the-fly training. The paper reported training in under five minutes and autonomous driving at up to 20 miles per hour on the tested roads. It also generated 14 shifted and rotated variants of each real image to teach recovery steering.[4] |
| 2000 | Algorithms for Inverse Reinforcement Learning | Ng and Russell formalized algorithms for recovering rewards under which demonstrated behavior is optimal and emphasized the ambiguity of the inverse problem.[5] |
| 2004 | Apprenticeship Learning via Inverse Reinforcement Learning | Abbeel and Ng matched expert feature expectations to obtain performance close to the expert under an unknown linear reward model.[6] |
| 2008 | Maximum Entropy Inverse Reinforcement Learning | Ziebart and colleagues introduced a globally normalized maximum-entropy trajectory model and applied it to route choice and destination prediction from taxi GPS data.[7] |
| 2010 | Efficient Reductions for Imitation Learning | Ross and Bagnell analyzed compounding error from treating sequential imitation as independent supervised prediction and proposed interactive reductions.[8] |
| 2011 | DAgger | Ross, Gordon, and Bagnell reduced imitation to no-regret online learning with iterative dataset aggregation on learner-visited states.[9] |
| 2016 | GAIL | Ho and Ermon developed adversarial occupancy-measure matching for model-free imitation from expert trajectories and learner rollouts.[10] |
| 2017 | DART | Laskey and colleagues optimized noise injected during demonstrations to collect recovery behavior without repeatedly executing an unassisted learner for labeling.[12] |
| 2018 | AIRL | Fu, Luo, and Levine introduced structured adversarial reward learning and evaluated reward transfer under changed dynamics.[11] |
| 2019 | Imitation from observation alone | Sun and colleagues gave a formal observation-only setting and a model-free distribution-matching algorithm with a sample-efficiency analysis.[24] |
| 2022 | Cross-embodiment and internet video | XIRL learned progress representations across embodiments, while Video PreTraining inferred action labels for large-scale Minecraft videos using inverse dynamics.[25][26] |
| 2023; expanded Diffusion Policy journal article in 2025 | ACT and Diffusion Policy | Sequence models for action chunks and conditional diffusion became strong reported approaches for visuomotor behavior cloning on the evaluated manipulation tasks.[15][16] |
| 2023 to 2025 | Cross-robot and vision-language-action policies | RT-1, RT-2, Open X-Embodiment, OpenVLA, pi0, and GR00T N1 investigated scaling demonstrated robot behavior across tasks, robots, and pretrained vision-language representations.[17][18][19][20][21][22] |
| 2025 | Broader data and continuous-control theory | Latent Diffusion Planning used action-free and suboptimal data through separate latent planning and inverse-dynamics modules, while new theory established stronger negative examples for smooth deterministic imitation in continuous control.[14][23] |

The two ALVINN entries are important to distinguish. The 1988 paper's training set was simulated, so it should not be described as a system trained by watching a human driver. The 1990 paper explicitly records a human driver's steering command as the desired output during real-time training. Both are relevant to the development of learned sensorimotor control, but they support different historical claims.[3][4]

## Modern robot learning

Modern robot imitation combines larger datasets with architectures that condition on images and language and predict sequences of actions. The term [vision-language-action model](https://aiwiki.ai/wiki/vision_language_action_model) is used for models that integrate visual observations, language instructions, and robot actions, often by adapting a pretrained vision-language model. The papers below report different datasets, embodiments, tasks, and evaluation protocols, so their headline numbers are not directly comparable.[18][20][21][22]

### Task-specific sequence policies

ACT and Diffusion Policy are frequently used as task-level policy architectures. ACT predicts a latent-conditioned action chunk with a transformer and smooths overlapping predictions. Diffusion Policy samples an action sequence through iterative conditional denoising. Both use closed-loop visual observations, but their action representations and inference costs differ. Their original papers focus on manipulation benchmarks and physical robot tasks rather than claiming universal performance across robot types.[15][16]

### RT-1 and RT-2

RT-1 is a language-conditioned robot transformer trained on a dataset collected over 17 months with 13 robots, about 130,000 episodes, and more than 700 tasks. The paper reports evaluation over 3,000 real-world trials and analyzes generalization to new tasks, distractors, and backgrounds. Its 35-million-parameter architecture tokenizes image, instruction, and robot-action information for real-time control.[17]

RT-2 co-fine-tunes vision-language models on web vision-language tasks and robot trajectories. Robot actions are represented as text tokens so that action prediction and language generation share an output format. The paper reports 6,000 evaluation trials and improvements on novel objects, commands, and semantic reasoning tests. These experiments support transfer within the authors' setups; they do not establish that web pretraining supplies reliable physical reasoning for unrestricted environments.[18]

### Open X-Embodiment and OpenVLA

The Open X-Embodiment collaboration standardized data from 22 robot embodiments contributed by 21 institutions, with 527 skills and 160,266 tasks reported in the paper. RT-X experiments trained variants of RT-1 and RT-2 on data from multiple robots and tested whether pooled experience improved policies on participating platforms. The work supplies evidence of positive cross-robot transfer in the reported evaluations while also exposing the challenge of reconciling different observations and action spaces.[19]

OpenVLA is a 7-billion-parameter open-source vision-language-action model built from a Llama 2 language model and DINOv2 and SigLIP visual features. It was trained on 970,000 real-world robot demonstration episodes from Open X-Embodiment. The paper reports an absolute task-success improvement of 16.5 percentage points over RT-2-X across its 29-task evaluation. That comparison is specific to the selected tasks, embodiments, implementations, and metric.[20]

### Flow and diffusion action models

The pi0 paper adds an action expert that produces continuous robot actions through flow matching on top of a pretrained vision-language model. Its pretraining mixture covers seven robot configurations and 68 tasks, and the paper evaluates direct prompting and fine-tuning on tasks including laundry folding, table cleaning, and box assembly. The work was first posted as a preprint in October 2024 and published at Robotics: Science and Systems in 2025. Its claims should be attributed to its reported experiments rather than treated as independent certification.[21]

GR00T N1 uses a dual-system architecture in which a vision-language module interprets observations and instructions and a diffusion-transformer module generates motor actions. Its training mixture includes real robot trajectories, human videos, and synthetic data. The authors report comparisons on simulation benchmarks and deployment on a Fourier GR-1 humanoid for language-conditioned bimanual manipulation. GR00T N1 was released as a 2025 preprint, so statements about its performance should retain that status and scope.[22]

### Learning from heterogeneous data

Large robot datasets often contain action-free video, suboptimal behavior, and different robot control spaces. Latent Diffusion Planning separates a latent video planner from an inverse-dynamics model. In the paper, this lets the planner use action-free demonstrations and the inverse model use suboptimal data, with evaluation on simulated visual manipulation tasks. The method demonstrates one way to use heterogeneous data; it does not show that arbitrary human video can be converted into executable robot actions.[23]

Pooling imperfect demonstrations also requires selecting or weighting useful behavior. Yue and colleagues evaluate a state-based data-selection method with lightweight behavior cloning across 21 offline-imitation benchmarks and report improvements on 20 of them. As with other benchmark results, the comparison depends on the datasets, baselines, and quality assumptions used in that study.[31]

## Applications

### Autonomous navigation and driving

ALVINN is an early documented application of neural control to road following. Its 1988 version learned from simulated road scenes, while the 1990 system learned from a human driver's steering and augmented images to include recovery behavior. The later procedure illustrates a problem that remains central: a good driver rarely demonstrates how to recover from every off-center position, so the training distribution must be broadened deliberately.[3][4]

Driving research often combines imitation with maps, planning, simulation, constraints, or reward optimization. The presence of a behavior-cloned component does not mean that an entire deployed driving stack is trained only through imitation. Claims about a commercial system's internal training method require direct documentation from that system and should not be inferred from the general literature.

### Robot manipulation and locomotion

Robot learning is the largest experimental domain in the cited literature. Demonstrations can provide dense supervision for contact-rich tasks for which sparse reward would make exploration difficult. ACT and Diffusion Policy study bimanual and visuomotor manipulation, while RT-1, Open X-Embodiment, OpenVLA, pi0, and GR00T N1 study broader task and embodiment mixtures.[15][16][17][19][20][21][22]

Abbeel and colleagues' autonomous-helicopter work used demonstrations and learned models to perform aerobatic maneuvers. The system segmented demonstrations, aligned repeated trajectories, learned a target trajectory, and used model-based control and reinforcement-learning techniques. It is an example of apprenticeship learning embedded in a larger control pipeline, not pure end-to-end behavior cloning.[27]

### Character animation

DeepMimic trains physics-based character controllers to imitate reference motion clips while also satisfying task objectives. The method combines an imitation objective with task rewards and uses reinforcement learning. It produced simulated skills such as running, backflips, and martial-arts motions in the reported environments. The application illustrates how imitation can specify style while reward terms specify task completion.[28]

### Games

AlphaStar used supervised learning from human StarCraft II data to initialize agents before reinforcement learning through league training. The final system therefore should not be described as a behavior-cloned policy alone. Demonstrations supplied a behavioral prior and helped define the early policy distribution, while self-play optimization produced later strategies.[29]

Video PreTraining applied semi-supervised imitation to Minecraft. An inverse-dynamics model inferred keyboard and mouse actions from online videos, after which a policy learned from the inferred action labels. The authors then fine-tuned policies with further imitation and reinforcement learning, including for tasks requiring long action sequences. This separates the contribution of scalable video-derived pretraining from subsequent task optimization.[26]

## Research boundaries

Imitation learning does not provide a single answer to reward specification, exploration, and safe deployment. Direct cloning avoids reward design but can reproduce flawed choices and fail under covariate shift. IRL infers an objective but is ambiguous without assumptions. Interactive imitation improves coverage but requires expert access and potentially risky rollouts. Generative sequence models can represent complex action distributions but do not by themselves validate behavior outside the data.[5][9][11][14][16]

Several active questions are empirical rather than settled facts: how data quality and diversity scale across embodiments, when web video supplies useful physical knowledge, how to evaluate long-horizon generalization, and how generative action models affect continuous-control error. Results from Open X-Embodiment and recent vision-language-action models provide evidence on selected robot suites, while recent theory supplies counterexamples to overly broad guarantees. Neither establishes a universal scaling law for imitation learning.[14][19][20][21][22]

A precise account of an imitation system should therefore state its demonstration source, observation and action spaces, interaction budget, expert-query requirements, reward use, policy class, evaluation distribution, and failure criteria. Without these details, labels such as "behavior cloning", "foundation model", or "learning from video" are too broad to determine what the system learned or what its results establish.

## See also

- [Machine learning](https://aiwiki.ai/wiki/machine_learning)
- [Deep learning](https://aiwiki.ai/wiki/deep_learning)
- [Autonomous driving](https://aiwiki.ai/wiki/autonomous_driving)
- [Foundation models](https://aiwiki.ai/wiki/foundation_models)

## References

[1] Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., and Peters, J. (2018). "An Algorithmic Perspective on Imitation Learning." Foundations and Trends in Robotics, 7(1-2), 1-179. https://doi.org/10.1561/2300000053

[2] Argall, B. D., Chernova, S., Veloso, M., and Browning, B. (2009). "A Survey of Robot Learning from Demonstration." Robotics and Autonomous Systems, 57(5), 469-483. https://doi.org/10.1016/j.robot.2008.10.024

[3] Pomerleau, D. A. (1988). "ALVINN: An Autonomous Land Vehicle in a Neural Network." Advances in Neural Information Processing Systems 1. https://papers.nips.cc/paper_files/paper/1988/hash/812b4ba287f5ee0bc9d43bbf5bbe87fb-Abstract.html

[4] Pomerleau, D. A. (1990). "Rapidly Adapting Artificial Neural Networks for Autonomous Navigation." Advances in Neural Information Processing Systems 3. https://papers.nips.cc/paper/1990/hash/248e844336797ec98478f85e7626de4a-Abstract.html

[5] Ng, A. Y., and Russell, S. J. (2000). "Algorithms for Inverse Reinforcement Learning." Proceedings of the 17th International Conference on Machine Learning, 663-670. https://ai.stanford.edu/~ang/papers/icml00-irl.pdf

[6] Abbeel, P., and Ng, A. Y. (2004). "Apprenticeship Learning via Inverse Reinforcement Learning." Proceedings of the 21st International Conference on Machine Learning. https://doi.org/10.1145/1015330.1015430

[7] Ziebart, B. D., Maas, A. L., Bagnell, J. A., and Dey, A. K. (2008). "Maximum Entropy Inverse Reinforcement Learning." Proceedings of the 23rd AAAI Conference on Artificial Intelligence, 1433-1438. https://www.cs.cmu.edu/~bziebart/publications/maximum-entropy-inverse-reinforcement-learning.html

[8] Ross, S., and Bagnell, J. A. (2010). "Efficient Reductions for Imitation Learning." Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 661-668. https://proceedings.mlr.press/v9/ross10a.html

[9] Ross, S., Gordon, G. J., and Bagnell, J. A. (2011). "A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning." Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, 627-635. https://proceedings.mlr.press/v15/ross11a.html

[10] Ho, J., and Ermon, S. (2016). "Generative Adversarial Imitation Learning." Advances in Neural Information Processing Systems 29. https://papers.nips.cc/paper_files/paper/2016/hash/cc7e2b878868cbae992d1fb743995d8f-Abstract.html

[11] Fu, J., Luo, K., and Levine, S. (2018). "Learning Robust Rewards with Adversarial Inverse Reinforcement Learning." International Conference on Learning Representations. https://openreview.net/forum?id=rkHywl-A-

[12] Laskey, M., Lee, J., Fox, R., Dragan, A., and Goldberg, K. (2017). "DART: Noise Injection for Robust Imitation Learning." Proceedings of the 1st Conference on Robot Learning, 143-156. https://proceedings.mlr.press/v78/laskey17a.html

[13] Swamy, G., Choudhury, S., Bagnell, J. A., and Wu, S. (2022). "Causal Imitation Learning under Temporally Correlated Noise." Proceedings of the 39th International Conference on Machine Learning, 20877-20890. https://proceedings.mlr.press/v162/swamy22a.html

[14] Simchowitz, M., Pfrommer, D., and Jadbabaie, A. (2025). "The Pitfalls of Imitation Learning When Actions Are Continuous." Proceedings of the Thirty-Eighth Conference on Learning Theory, 5248-5351. https://proceedings.mlr.press/v291/simchowitz25a.html

[15] Zhao, T. Z., Kumar, V., Levine, S., and Finn, C. (2023). "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware." Robotics: Science and Systems XIX. https://roboticsconference.org/2023/program/papers/016/

[16] Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., and Song, S. (2025). "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." The International Journal of Robotics Research, 44(10-11), 1684-1704. https://doi.org/10.1177/02783649241273668

[17] Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., et al. (2023). "RT-1: Robotics Transformer for Real-World Control at Scale." Robotics: Science and Systems XIX. https://roboticsconference.org/2023/program/papers/025/

[18] Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., et al. (2023). "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control." arXiv:2307.15818. https://arxiv.org/abs/2307.15818

[19] Open X-Embodiment Collaboration et al. (2024). "Open X-Embodiment: Robotic Learning Datasets and RT-X Models." 2024 IEEE International Conference on Robotics and Automation, 6892-6903. https://doi.org/10.1109/ICRA57147.2024.10611477

[20] Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., et al. (2025). "OpenVLA: An Open-Source Vision-Language-Action Model." Proceedings of the 8th Conference on Robot Learning, 2679-2713. https://proceedings.mlr.press/v270/kim25c.html

[21] Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., et al. (2025). "pi0: A Vision-Language-Action Flow Model for General Robot Control." Robotics: Science and Systems XXI. https://roboticsconference.org/2025/program/papers/10/

[22] NVIDIA. (2025). "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots." arXiv:2503.14734. https://arxiv.org/abs/2503.14734

[23] Xie, A., Rybkin, O., Sadigh, D., and Finn, C. (2025). "Latent Diffusion Planning for Imitation Learning." Proceedings of the 42nd International Conference on Machine Learning, 68710-68724. https://proceedings.mlr.press/v267/xie25h.html

[24] Sun, W., Vemula, A., Boots, B., and Bagnell, J. A. (2019). "Provably Efficient Imitation Learning from Observation Alone." Proceedings of the 36th International Conference on Machine Learning, 6036-6045. https://proceedings.mlr.press/v97/sun19b.html

[25] Zakka, K., Zeng, A., Florence, P., Tompson, J., Bohg, J., and Dwibedi, D. (2022). "XIRL: Cross-Embodiment Inverse Reinforcement Learning." Proceedings of the 5th Conference on Robot Learning, 537-546. https://proceedings.mlr.press/v164/zakka22a.html

[26] Baker, B., Akkaya, I., Zhokov, P., Huizinga, J., Tang, J., Ecoffet, A., et al. (2022). "Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos." Advances in Neural Information Processing Systems 35. https://arxiv.org/abs/2206.11795

[27] Abbeel, P., Coates, A., and Ng, A. Y. (2010). "Autonomous Helicopter Aerobatics through Apprenticeship Learning." The International Journal of Robotics Research, 29(13), 1608-1639. https://doi.org/10.1177/0278364910371999

[28] Peng, X. B., Abbeel, P., Levine, S., and van de Panne, M. (2018). "DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills." ACM Transactions on Graphics, 37(4), Article 143. https://doi.org/10.1145/3197517.3201311

[29] Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., et al. (2019). "Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning." Nature, 575, 350-354. https://doi.org/10.1038/s41586-019-1724-z

[30] Brown, D. S., Goo, W., and Niekum, S. (2020). "Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations." Proceedings of the 3rd Conference on Robot Learning, 330-359. https://proceedings.mlr.press/v100/brown20a.html

[31] Yue, S., Liu, J., Hua, X., Ren, J., Lin, S., Zhang, J., and Zhang, Y. (2024). "How to Leverage Diverse Demonstrations in Offline Imitation Learning." Proceedings of the 41st International Conference on Machine Learning, 58037-58067. https://proceedings.mlr.press/v235/yue24c.html