# Embodied AI

> Source: https://aiwiki.ai/wiki/embodied_ai
> Updated: 2026-07-28
> Fact-checked: 2026-07-28
> Categories: Artificial Intelligence, Deep Learning, Reinforcement Learning, Robotics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "Embodied AI." aiwiki.ai, 28 Jul 2026. https://aiwiki.ai/wiki/embodied_ai
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

Embodied AI is an area of [artificial intelligence](https://aiwiki.ai/wiki/artificial_intelligence) concerned with agents whose perception, action, learning, and reasoning are coupled through interaction with an environment. An embodied agent receives observations through a particular set of sensors, can change the environment through a particular set of actions, and experiences the consequences through a feedback loop. Its body may be a physical machine or a simulated body used to study situated behavior. What matters operationally is that the agent cannot treat the environment as a fixed input file: its actions affect what it encounters and what it can observe next.

The field overlaps with [AI agents](https://aiwiki.ai/wiki/ai_agents) and [robotics](https://aiwiki.ai/wiki/robotics), but the terms are not interchangeable. A fixed industrial controller can operate a robot without learning a broad representation of its surroundings. Conversely, a virtual agent navigating an interactive three-dimensional world can be embodied for an experiment even though no physical robot is present. The related term [physical AI](https://aiwiki.ai/wiki/physical_ai) is often reserved for systems acting in the physical world, while embodied AI is also used for simulated agents and for research on the general relation among body, task, and environment.

Embodiment introduces constraints that are easy to hide in static-data evaluations. Sensors are incomplete and noisy; actions take time and can have uncertain effects; contact changes objects and the agent itself; control must meet physical timing limits; and a failure may damage hardware or harm a person. These facts do not establish that embodiment is necessary for every kind of intelligence. They do make embodiment central to any claim about an agent that must reliably perceive and act in an environment. Modern work combines control, planning, learning from demonstrations, reinforcement learning, simulation, world models, and vision-language models rather than following one universal architecture.

## Definition and scope

An embodied AI experiment is defined by more than a model name. At minimum, it specifies an environment, an embodiment, an observation interface, an action interface, a task or objective, and a protocol for resetting and evaluating the system. The embodiment includes morphology, kinematic and dynamic constraints, sensor placement, actuator limits, timing, and any low-level controller between a learned policy and the hardware.

This operational definition avoids two common ambiguities. First, embodiment does not require a humanoid form. A wheeled robot, drone, robot arm, soft gripper, quadruped, autonomous vehicle, or simulated avatar can each have an embodiment suited to its environment. Second, embodiment is not equivalent to a claim about consciousness or human cognition. Research can study the measurable consequences of sensorimotor coupling without resolving philosophical disputes about mind or subjective experience.

The term has several related uses:

| Use | Central question | Typical experimental object |
| --- | --- | --- |
| Embodied AI | How can an agent perceive, decide, learn, and act through a body in an environment? | Physical robot or interactive simulated agent |
| Embodied intelligence | How do body, materials, controller, and environment jointly produce adaptive behavior? | Robot, soft machine, or evolved morphology |
| Embodied cognition | How does bodily interaction contribute to cognition? | Human, animal, cognitive model, or robot |
| [Cognitive robotics](https://aiwiki.ai/wiki/cognitive_robotics) | How can robots integrate perception, memory, reasoning, planning, and action? | Physical or simulated robot |
| Physical AI | How can AI systems model and act in the physical world? | Robot, vehicle, industrial system, or physical-world model |

These uses overlap, and no single taxonomy governs every paper. A technical article therefore needs to state which meaning it adopts. Here, embodied AI refers to artificial agents operating in a closed perception-action loop. Stronger claims about whether cognition is constituted by the body are treated as related theory, not as part of the definition.

### Embodiment, situation, and grounding

An agent is situated when its behavior depends on its current relation to an environment. It is embodied when the channels through which it senses and acts are constrained by a body or body-like interface. The distinction matters because two agents placed in the same scene may face different problems. A fixed camera above a table, a camera on a robot wrist, and tactile sensors in a gripper expose different observations and blind spots. A wheeled base and a legged base make different regions reachable. A parallel-jaw gripper and a dexterous hand have different action spaces.

The symbol-grounding problem asks how symbols can acquire meaning that is not supplied only by other symbols. Harnad proposed that elementary symbols be grounded in nonsymbolic representations tied to sensory categories.[1] Sensorimotor interaction is one route to such [grounding](https://aiwiki.ai/wiki/grounding), but a robot's possession of sensors and motors does not by itself show that every internal representation is grounded or semantically correct. Grounding is task, data, and interface dependent.

Embodied cognition is a broader research program spanning philosophy, psychology, neuroscience, linguistics, and robotics. Some versions argue that bodily processes contribute constitutively to cognition; others make the narrower claim that bodily constraints are useful variables in explanations. Embodied AI draws motivation from this literature but can be evaluated through engineering criteria such as task success, safety, transfer, and resource use.

### Physical and simulated embodiment

Physical embodiment exposes an agent to real sensing, contact, latency, wear, energy limits, and safety consequences. It also makes data collection slow and expensive. Simulated embodiment makes it possible to run controlled experiments, parallelize experience, reproduce initial conditions, and test failures without damaging hardware. Habitat, for example, was explicitly designed to study configurable virtual agents and embodied tasks in scanned three-dimensional environments.[2]

A simulated agent is not physically embodied in the same sense as a robot, but it can still be operationally embodied: it has a viewpoint, action set, motion constraints, and a history of interaction. Results remain claims about that simulator and protocol until transfer is shown. A photorealistic image does not guarantee accurate contact, friction, deformable objects, sensor noise, actuator dynamics, or human behavior.

## Historical development

Embodied AI did not replace one earlier paradigm at a single date. Its development combines long-standing robotics, control theory, active perception, decision theory, cognitive science, and machine learning.

### Integrated perception, planning, and action

SRI's Shakey project ran from 1966 through 1972. Shakey perceived a simplified environment, built a model, planned routes and object rearrangements, and executed actions through a mobile platform.[3] The project is important because it connected symbolic planning to sensing and actuation. It also exposed a difficulty that remains current: an internal plan must be revised when observations or action outcomes differ from its assumptions.

Active perception developed the complementary idea that sensing is itself subject to control. Bajcsy described perception as an active process in which an agent selects sensing actions based on the task, the sensor, and uncertainty.[4] Moving a camera to reduce occlusion, touching an object to estimate compliance, or walking around a scene to reveal a hidden region are not preprocessing steps outside intelligence. They are actions chosen to obtain useful information.

The symbol-grounding debate and embodied-cognition research added a conceptual challenge to purely formal manipulation of symbols.[1] These traditions influenced embodied AI, but they did not yield one mandatory software architecture. Modern systems may use explicit maps, learned latent state, language symbols, geometric models, or combinations of them.

### Behavior-based robotics

Rodney Brooks's 1991 paper "Intelligence without Representation" argued for building complete agents incrementally and decomposing control into parallel activity-producing systems directly coupled to the world.[5] Brooks contrasted this approach with architectures that depended on a single detailed representation before acting. The work helped establish behavior-based robotics and demonstrated that useful real-time behavior could arise from layered reactive control.

This historical argument is sometimes summarized too strongly. It did not prove that internal state, prediction, or planning is always unnecessary. Current embodied systems often combine fast reactive controllers with slower planners and learned models. The durable contribution is the insistence that intelligence claims be tested in a complete perception-action system, not only in a detached representation.

### Learning-based embodied systems

Robotic learning long predates current foundation models. Reinforcement-learning research treated motor behavior as sequential decision-making under uncertain dynamics, while learning-from-demonstration research studied how policies could be derived from examples supplied by people.[6][7] Deep visuomotor work later showed that visual features and motor policies could be trained jointly for selected real-world manipulation tasks, reducing some hand-engineered boundaries between perception and control.[8]

The growth of [deep learning](https://aiwiki.ai/wiki/deep_learning) changed the scale of perception and policy learning, but it did not remove classical robotics problems. State estimation, calibration, kinematics, collision checking, trajectory generation, impedance, and feedback control remain parts of many deployed systems. The research question shifted from whether learning should be used to which components should be learned, from what data, under which constraints, and with what fallback behavior.

### Selective milestones

| Period | Development | Lasting significance |
| --- | --- | --- |
| 1966-1972 | Shakey at SRI | Integrated perception, world modeling, planning, and mobile action |
| 1980s | Active perception and behavior-based robotics | Treated sensing and reactive control as parts of action selection |
| 1990s | Symbol grounding, situated AI, and formal partial-observability work | Clarified the relation among observations, symbols, uncertainty, and action |
| 2000s | Learning from demonstration and robot reinforcement learning | Made policy acquisition from people and interaction a central research topic |
| 2010s | Deep visuomotor policies and high-throughput simulation | Supported learning from images and large simulated experience |
| 2017-2019 | Domain-randomized sim-to-real demonstrations | Showed transfer in bounded perception and control settings while exposing the reality gap |
| 2019 onward | Habitat, AI2-THOR, RLBench, and related environments | Standardized interactive tasks and scalable evaluation |
| 2022 onward | Large robot datasets and transformer policies | Shifted attention toward multi-task, multi-robot pretraining and adaptation |
| 2023 onward | Vision-language-action models | Connected semantic visual-language pretraining with robot action prediction |
| 2025-2026 | World-model and physical-reasoning benchmarks | Increased scrutiny of whether generated or latent predictions are physically executable |

The table identifies research transitions, not a ranking of systems. Many methods coexist because they solve different parts of the embodied problem.

## The agent-environment loop

The closed loop is the central unit of analysis. At time step `t`, an environment has a state that may not be directly observable. The agent receives an observation, chooses an action according to a [policy](https://aiwiki.ai/wiki/policy), and the environment changes. A general stochastic formulation is:

$$
s_{t+1} \sim T(s_{t+1}\mid s_t,a_t), \qquad
o_t \sim O(o_t\mid s_t), \qquad
a_t \sim \pi(a_t\mid h_t).
$$

Here `s_t` is the environment state, `o_t` is the observation, `a_t` is the action, and `h_t` is the information available to the policy, such as recent observations, actions, goals, and memory. The transition model `T` and observation model `O` can be known, learned, approximate, or implicit. The notation does not require a reward function: a task may instead be specified through constraints, demonstrations, a planner, language, or a goal state.

### Partial observability and memory

Embodied environments are usually partially observable. A camera cannot see behind an obstacle; a force sensor measures only certain contacts; odometry drifts; and the internal state of another person is not directly available. A policy based only on the latest image may therefore confuse states that require different actions.

Partially observable Markov decision processes formalize this problem by separating hidden state from observations and maintaining a belief or information state over what may be true.[9] Exact POMDP solution is intractable for many realistic problems, so embodied systems use approximations: geometric estimators, recurrent networks, map memories, object-centric state, particle filters, or learned latent representations. Calling a policy "memory based" is not enough to establish that it stores the right information; evaluation must test delayed consequences, re-observation, and recovery.

[Sensor fusion](https://aiwiki.ai/wiki/sensor_fusion) combines evidence from cameras, depth sensors, lidar, inertial units, joint encoders, microphones, force-torque sensors, and other modalities. Fusion can occur at raw-signal, feature, state-estimate, or decision levels. Each choice makes assumptions about synchronization, calibration, noise, and missing data. More sensors can reduce ambiguity, but they also create more failure modes and timing dependencies.

### Action spaces and control interfaces

An action can mean a high-level skill, a Cartesian end-effector displacement, a target joint position, a joint velocity, a torque, or a discrete movement in a simulator. These are not equivalent outputs. A high-level command may pass through a planner and a feedback controller before reaching motors. A torque policy has more direct influence over contact but must satisfy a faster and more demanding control loop.

Action frequency and latency are part of the task. A language model that updates once per second cannot by itself stabilize a fast dynamic system. Hierarchical designs often use a slower module for task interpretation or planning and a faster controller for motion. [Model Predictive Control](https://aiwiki.ai/wiki/model_predictive_control), impedance control, operational-space control, and learned low-level policies can each serve this role depending on the robot and task.

The action interface also determines transfer. Two arms may both accept end-effector targets but differ in reach, joint limits, grippers, compliance, and collision geometry. A policy trained on one action convention cannot be assumed to control another merely because both data records contain vectors of the same length.

### Feedback and error correction

Open-loop execution predicts an action sequence and performs it without using subsequent observations to correct the plan. Closed-loop execution observes during the task and can respond to deviations. Most useful embodied behavior needs feedback because objects slip, people move, and estimated poses are imperfect.

Feedback can operate at several levels:

- a motor controller corrects joint or force error;
- a state estimator updates pose and velocity;
- a local planner avoids a newly detected obstacle;
- a skill policy adjusts a grasp from visual or tactile observations;
- a task executive retries, selects another skill, or asks a person for help;
- a high-level reasoner revises the task plan after detecting failure.

These levels can fail independently. A system may choose the right task step and execute it poorly, or execute a motion accurately toward the wrong object. Evaluation should therefore separate perception, planning, control, and recovery where possible instead of reporting only one end-to-end success number.

### Active perception and affordances

Perception and action are coupled when the agent can act to improve its information. Active perception includes selecting a viewpoint, changing illumination, touching an object, opening a container, or moving a distractor. The benefit is task specific. A view that is useful for recognizing an object may be poor for estimating a grasp.

An affordance describes an action possibility relative to an agent and environment, such as whether an object can be grasped with a particular gripper or whether a gap can be traversed by a particular body. Affordances are not fixed visual labels independent of embodiment. The same handle may be reachable for one arm and not another; a surface may support one robot but not a heavier platform.

[Tactile sensing](https://aiwiki.ai/wiki/tactile_sensing) is especially important after contact, when cameras may be occluded and small force changes determine whether an object slips or deforms. Tactile observations can support grasp adjustment, material estimation, insertion, and safety monitoring. They also introduce calibration, durability, bandwidth, and sensor-placement constraints.

### Morphology and embodied computation

Morphology affects what the controller must compute. Passive compliance can absorb impact; a tendon arrangement can coordinate joints; wheel geometry can simplify balance; and a soft gripper can conform to uncertain object shape. Research on embodied intelligence treats materials, morphology, sensing, actuation, and control as a co-design problem.[10]

The phrase "the body computes" should be used carefully. A mechanical structure can transform forces and constrain motion without executing a digital algorithm. Its dynamics can reduce sensing or control demands, but they can also introduce hysteresis, wear, or difficult-to-model behavior. Morphological benefit must be measured for a specified task, environment, and controller.

## Architectures and control

Embodied systems are often described as modular or end to end, but most practical architectures lie between those poles. A modular system exposes intermediate objects such as poses, maps, trajectories, grasps, or skills. An end-to-end learned policy may map observations more directly to actions, but preprocessing, actuator control, safety filtering, and task logic can still remain outside the learned network.

### Modular pipelines

A conventional mobile-robot pipeline may estimate pose, construct a map, choose a route, generate a collision-free local trajectory, and send velocity targets to a controller. A manipulation pipeline may detect objects, estimate their poses, choose a grasp, plan a collision-free arm motion, and regulate position or force during contact. [Simultaneous Localization and Mapping](https://aiwiki.ai/wiki/slam) and [motion planning](https://aiwiki.ai/wiki/motion_planning) are distinct components even when they share sensor data.

Modularity offers inspectable interfaces and lets engineers substitute or verify components. Its weakness is that errors can propagate across interfaces. A perception system may output a confident but wrong pose; a planner may assume a static environment; a controller may receive a trajectory that violates a contact constraint. Optimizing each module on its own metric does not guarantee task success.

Learned components can appear at any point in a modular pipeline. A network can estimate depth, segment objects, predict traversability, rank grasps, approximate dynamics, choose skills, or correct residual control error. "Learning based" therefore does not imply that geometry or control theory has been discarded.

### Reactive and deliberative layers

Reactive control maps current information to action with little explicit lookahead. It is useful when rapid response matters or the environment supplies reliable feedback. Deliberative control predicts future states or searches over action sequences. It is useful for constraints, long horizons, and tasks where an early action changes later possibilities.

Hierarchical systems assign these roles to different time scales. A task planner may choose "open drawer", a skill policy may generate a sequence of end-effector targets, and a low-level controller may regulate contact forces. The hierarchy can reduce the sequence length seen by each component, but it introduces interfaces and failure-detection problems. If the lower-level skill does not achieve the state assumed by the planner, later steps can be invalid.

Hi Robot is one research example of this pattern. Its high-level vision-language component interprets complex instructions and feedback, while lower-level policies execute selected steps.[11] The result supports hierarchical decomposition in its tested settings; it does not establish that language reasoning alone supplies reliable motor control.

### World models and planning

A [world model](https://aiwiki.ai/wiki/world_model) predicts aspects of an environment or the consequences of candidate actions. It may operate on physical state, images, object relations, or a learned latent representation. A controller can use the model for planning, policy improvement, uncertainty estimation, or imagined training data.

World models have several different meanings:

| Model role | Input and prediction | Typical use |
| --- | --- | --- |
| Dynamics model | State and action to next state | Model-predictive control or trajectory optimization |
| Observation model | State to expected sensor reading | State estimation and belief update |
| Latent predictive model | Encoded history and action to future representation | Planning or representation learning |
| Generative video model | Images and conditions to future frames | Data generation, scenario exploration, or visual planning |
| Semantic scene model | Objects, relations, and actions to symbolic consequences | Task planning and explanation |

Predictive accuracy must be measured in the variables that matter for action. A visually plausible future may still put a gripper on the wrong side of an object or violate contact dynamics. RoboWM-Bench reported exactly this gap in evaluated video world models: generated behavior could look plausible while containing spatial errors, unstable contacts, or nonphysical deformations that made execution fail.[12]

DreamerV3 showed that a learned latent world model could support one algorithm across a wide range of simulated control domains without domain-specific hyperparameter changes.[13] V-JEPA 2 later explored video pretraining followed by action-conditioned post-training for selected robot planning tasks.[14] These results show several ways predictions can support action, but neither is evidence that one learned model accurately captures all real-world physics.

## Learning paradigms

Embodied AI uses several learning signals. They differ in who supplies the objective, where data come from, and whether the learner can explore.

### Reinforcement learning

[Reinforcement learning](https://aiwiki.ai/wiki/reinforcement_learning) learns a policy from interaction and a reward or return objective. It is attractive when the desired behavior is easier to evaluate than to demonstrate or program. Locomotion, grasp refinement, and control under uncertain dynamics can fit this formulation.

Robotics makes reinforcement learning difficult. Physical experience is slow; exploration can damage hardware; rewards can be sparse or misspecified; observations are partial; and resets may require a person. Kober, Bagnell, and Peters emphasized that robot reinforcement learning must confront high-dimensional continuous control, sample cost, model uncertainty, and safety.[6] Simulation, demonstrations, prior models, curricula, offline data, and constrained exploration are therefore commonly combined with online learning.

A reward is not a complete task specification. A policy may exploit an unintended shortcut that scores well without achieving the intended outcome. Reward reports should state termination rules, penalties, success detectors, reset conditions, and whether privileged simulator state was used. A reward based on object pose is different from one inferred from camera images, even if the policy receives the same observations.

### Learning from demonstration

[Imitation Learning](https://aiwiki.ai/wiki/imitation_learning) uses examples of behavior supplied by a person, another policy, or an optimizer. Demonstrations may be collected through kinesthetic teaching, joystick or virtual-reality [teleoperation](https://aiwiki.ai/wiki/teleoperation), instrumented tools, scripted experts, or existing logs. Argall and colleagues organized the field around who demonstrates, how observations and actions correspond between teacher and robot, how a policy is derived, and how performance is assessed.[7]

Demonstrations avoid unsafe random exploration and can communicate complex intent. They also contain limitations. The data may cover only successful behavior, so the learned policy does not know how to recover. A human and robot may have different viewpoints or bodies. Teleoperation latency can distort motion. A demonstrator may solve a task in several valid ways, making an average action invalid.

Behavior cloning treats action prediction as supervised learning and can compound errors when the learned policy visits states absent from the demonstration data. Interactive imitation methods can request corrections or collect examples at the learner's states. Offline policy learning can use fixed datasets, but its estimates are least reliable for actions poorly represented in the data.

### Model-based and self-supervised learning

Model-based methods learn or use dynamics and plan through them. They can reuse experience across objectives and inspect predicted outcomes, but model error compounds over a planning horizon. Uncertainty-aware planning, short-horizon replanning, ensembles, and residual corrections can reduce the effect without eliminating it.

Self-supervised learning obtains targets from the structure of observations rather than hand labels. An agent may predict future observations, temporal order, correspondence across cameras, masked features, contact events, or the result of an action. Video is abundant compared with robot action data, but ordinary video lacks the robot's action labels, proprioception, and exact forces. Learning "what tends to happen" from video is not the same as learning which motor command will cause it.

[Active Learning](https://aiwiki.ai/wiki/active_learning) can prioritize informative data, while curiosity or information-gain objectives can encourage exploration. These objectives require safety limits and can conflict with the external task. An informative collision is still unacceptable on a real platform.

### Hybrid learning and classical control

Many strong systems are hybrids:

- demonstrations initialize a policy, reinforcement learning refines it, and a safety controller constrains execution;
- simulation supplies broad experience, real data calibrate or adapt the policy, and online feedback corrects remaining mismatch;
- a language model selects a skill, a learned policy produces motion targets, and a classical controller tracks them;
- an analytical model handles known dynamics while a learned residual models unmodeled effects;
- a planner chooses a sequence and a learned value or success model ranks alternatives.

Hybrid architecture is not a compromise that can be evaluated without details. Performance depends on which module has authority, how uncertainty is passed between levels, and what happens when modules disagree.

## Task families

Embodied AI covers tasks with different observations, actions, horizons, and success conditions. A method that performs well in one family may not transfer to another.

### Navigation and exploration

[Robot navigation](https://aiwiki.ai/wiki/robot_navigation) asks an agent to move through an environment while avoiding hazards and reaching or discovering relevant locations. Point-goal navigation supplies coordinates; object-goal navigation specifies an object category; image-goal navigation supplies a target view; vision-and-language navigation supplies natural-language instructions. The 2018 Room-to-Room benchmark formalized language-conditioned navigation in previously unseen buildings using panoramic imagery.[15]

Navigation may use a known map, construct a map online, or operate without an explicit geometric map. It may receive depth, GPS-like coordinates, odometry, or only egocentric images. These differences change the problem substantially. An agent with perfect simulator localization is not solving the same problem as a physical robot estimating pose from noisy sensors.

Success alone can reward an unnecessarily long or collision-prone path. The navigation evaluation working group proposed reporting success weighted by path length and specifying generalization across scenes, goals, and sensor conditions.[16] Further measures may include collisions, energy, time, intervention, map quality, localization error, or instruction fidelity.

Exploration adds an information objective: the agent chooses where to move to map space, locate an object, answer a question, or reduce uncertainty. In embodied question answering, for example, the agent must navigate to gather evidence before answering. A correct answer reached through privileged scene access would not measure the intended embodied capability.

### Manipulation

[Robot manipulation](https://aiwiki.ai/wiki/robot_manipulation) includes grasping, placing, insertion, tool use, deformable-object handling, assembly, and in-hand manipulation. It couples vision, geometry, contact, force, and control. A task may be successful only if the right object reaches a goal state without excessive force or damage.

Manipulation is sensitive to details often omitted from model summaries:

- the gripper or hand geometry;
- the camera viewpoint and calibration;
- the control mode and frequency;
- object mass, friction, compliance, and pose;
- whether the object set was seen during training;
- tolerance used by the success detector;
- whether a human resets, repositions, or rescues the robot;
- whether timeouts and failed attempts are counted.

RLBench provides a standardized simulated suite for vision-guided manipulation research.[17] CALVIN emphasizes language-conditioned sequences of skills over longer horizons.[18] BEHAVIOR-1K broadens the target to 1,000 everyday activities involving many objects and physical properties, and its evaluation identifies long-horizon, contact-rich behavior as an open challenge.[19] The benchmarks complement rather than replace one another because they ask different questions.

### Locomotion and whole-body control

Legged locomotion requires balance, contact scheduling, disturbance recovery, and adaptation to terrain. Whole-body manipulation adds arm or hand tasks while the base moves or balances. The policy must coordinate several time scales and respect joint, torque, contact, and stability limits.

Simulation is especially useful because falls and impacts are expensive on hardware. Dynamics randomization can expose a policy to varying masses, friction, delay, and actuator response. Peng and colleagues demonstrated transfer for a bounded robotic control setup using randomized dynamics.[20] Such results depend on the simulated model, randomization distribution, robot, controller, and test conditions.

Humanoid form is one embodiment among many. A [Humanoid Robot](https://aiwiki.ai/wiki/humanoid_robot) may fit environments designed for people, but it adds high-dimensional control and balance constraints. Embodied intelligence should be judged by task requirements rather than resemblance to a human body.

### Mobile manipulation

Mobile manipulation combines navigation, reachability, grasping, and interaction. The base must stop where the arm can reach, the arm motion must account for the base and environment, and actions such as opening a door alter future navigation. Treating navigation and manipulation as independent episodes can hide these dependencies.

Long tasks also require memory and recovery. A system needs to know which subtasks succeeded, detect when an object was dropped or moved, and decide whether to retry, replan, or ask for help. Success rates measured on short, manually reset skills do not directly predict performance on an hour-long household task.

### Language, social interaction, and collaboration

Language can specify goals, constraints, preferences, and corrections. A language-conditioned policy must connect words to the current scene and to actions available to its embodiment. This is more than object recognition. "Put the cup where it will not spill" requires interpreting the instruction, estimating physical state, planning, and monitoring execution.

[Human-Robot Interaction](https://aiwiki.ai/wiki/human_robot_interaction) adds people whose motion, attention, intentions, and comfort matter. The robot may need to communicate uncertainty, negotiate shared space, yield control, or accept correction. A benchmark with scripted humans cannot represent every social or safety condition.

Language models can contribute semantic knowledge and task decomposition, but fluent explanations do not prove grounded perception or successful control. The high-level plan and physical execution need separate evaluation, especially when the system can generate a plausible verbal account after an unsuccessful action.

### Virtual embodied agents

Virtual agents support experiments that would be difficult to run physically. AI2-THOR provides interactive indoor scenes for navigation, object interaction, planning, and learning.[21] Habitat focuses on configurable agents, sensor suites, task definitions, and high-throughput simulation.[2] Game-like or generated worlds can expand visual and behavioral diversity.

Virtual embodiment is valuable when the research claim is clearly scoped. A discrete action such as "move forward" may hide collision avoidance and low-level control. A scanned environment may provide realistic images but no object interaction. A generated environment may contain visual inconsistency or implausible physics. Researchers need to state which layers the simulator supplies and which the agent controls.

## Simulation and transfer

[Simulation](https://aiwiki.ai/wiki/simulation) provides dynamics, rendering, sensors, scenes, and task logic at different levels of fidelity. It can generate large datasets, expose privileged state for training, support repeatable ablations, and accelerate unsafe or rare scenarios. It can also encode assumptions that a policy exploits.

### What simulators model

Physics engines numerically approximate motion, constraints, and contact. [MuJoCo](https://aiwiki.ai/wiki/mujoco), for example, was designed around generalized-coordinate dynamics and efficient contact computation for model-based control.[22] Scene simulators such as Habitat and AI2-THOR emphasize embodied perception and interaction in indoor environments.[2][21] No one platform is most faithful for every phenomenon.

Simulation fidelity is multidimensional:

| Dimension | Examples of mismatch |
| --- | --- |
| Geometry | Inaccurate shape, collision mesh, articulation, or clearance |
| Dynamics | Mass, inertia, friction, damping, compliance, backlash, or contact model |
| Sensors | Noise, blur, latency, exposure, missing returns, calibration, or occlusion |
| Actuation | Delay, saturation, deadband, thermal behavior, or controller implementation |
| Appearance | Lighting, material, texture, clutter, weather, or camera response |
| People | Motion, intent, reaction, social convention, and rare behavior |
| Task logic | Simplified success detector, reset, object state, or allowed intervention |

Higher visual realism can coexist with poor dynamics, and an accurate rigid-body model can coexist with unrealistic sensor images. Fidelity should be chosen for the claim being tested.

### The reality gap

The reality gap is the difference between training or evaluation conditions in simulation and the physical deployment environment. A policy can overfit to a renderer, exploit simulator timing, depend on exact state, or learn a contact strategy that fails under unmodeled friction.

[Sim-to-real transfer](https://aiwiki.ai/wiki/sim_to_real_transfer) uses several complementary methods:

- system identification estimates physical parameters and calibrates the simulator;
- domain randomization varies visual, physical, and timing parameters during training;
- [domain adaptation](https://aiwiki.ai/wiki/domain_adaptation) aligns representations or predictions across simulated and real data;
- residual learning corrects a model or controller using real experience;
- fine-tuning updates a simulated policy on physical demonstrations or interaction;
- online adaptation infers changing dynamics from recent history;
- privileged training uses simulator-only information for a teacher while a deployable student receives realistic observations.

Tobin and colleagues demonstrated visual domain randomization for a bounded object-localization and grasping system.[23] OpenAI's dexterous-hand work used automatic domain randomization and a hierarchical setup to transfer a Rubik's Cube manipulation system.[24] These are important case studies, not proof that randomization is sufficient for every robot or contact process.

Randomization that is too narrow excludes the real system. Randomization that is too broad can make learning inefficient or produce a conservative policy. The selected distribution is therefore part of the method and should be reported.

### Real-world validation

A sim-to-real claim needs physical evaluation on the intended observation and action interfaces. Useful reporting includes the number of trials, initial-state distribution, objects and environments, success definition, failure types, human interventions, safety stops, and hardware variation. A video montage without failed trials cannot establish reliability.

A [digital twin](https://aiwiki.ai/wiki/digital_twin) may track a particular physical system more closely than a generic simulator, but it also drifts as hardware wears or the environment changes. Calibration and monitoring are continuing processes, not one-time proof of equivalence.

Real data remain necessary for phenomena that are difficult to simulate or observe accurately. The question is how to allocate expensive physical experience: calibration, demonstrations, failure recovery, fine-tuning, validation, or online improvement.

## Generalist policies and foundation models

Large pretrained policies seek to reuse data across tasks, environments, and bodies. The goal is not merely a larger controller. It is a model that can acquire broad priors, accept varied task specifications, and adapt with less task-specific data than training from scratch.

[Foundation Models](https://aiwiki.ai/wiki/foundation_models) in robotics can be pretrained on robot trajectories, images, video, language, simulation, or combinations of these sources. The phrase "robot foundation model" does not guarantee a particular architecture, open release, or level of generality. Some models output actions directly; others produce representations, plans, rewards, value estimates, or synthetic data.

### Vision-language-action models

A [vision-language-action model](https://aiwiki.ai/wiki/vision_language_action_model) receives visual observations and language or another goal representation, then predicts robot actions or action chunks. It often starts from a [vision language model](https://aiwiki.ai/wiki/vision_language_model) whose semantic representations were learned from image-text data. Robot trajectories add the connection from observation and instruction to action.

RT-1 demonstrated a transformer policy trained on a large real-robot dataset covering many instructions for one fleet and environment family.[25] RT-2 co-fine-tuned vision-language and robot-action data and represented actions in a token-compatible form, showing selected transfer from web pretraining to robot tasks.[26] These works established important scaling patterns, but their reported success rates are tied to particular robots, task splits, data collection, and success criteria.

Action generation remains a robotics-specific problem. Discrete tokens fit language-model infrastructure but quantize continuous motion. Regression predicts direct values but may average incompatible behaviors. Diffusion or flow-matching decoders can represent multimodal continuous action distributions at added inference cost. Action chunking predicts several future commands at once, reducing repeated model calls but making within-chunk correction less immediate. The pi0 system combined a pretrained visual-language backbone with a flow-matching action expert for continuous action generation across several robot platforms.[27]

A [large language model](https://aiwiki.ai/wiki/large_language_model) can also sit above the control policy. It may translate an instruction into skills, query scene information, or revise a plan. This can make high-level behavior more flexible while leaving geometric feasibility and control to lower layers. Language plausibility must not be used as a proxy for successful physical execution.

### Data diversity and cross-embodiment learning

The Open X-Embodiment collaboration standardized datasets from multiple institutions and robot types and reported positive transfer for tested RT-X policies.[28] Its importance lies as much in data normalization as in scale. Robot records can differ in camera layout, control frequency, action coordinate frame, gripper convention, language annotation, and episode boundaries. Combining files does not automatically make the underlying actions commensurate.

DROID collected diverse real-robot manipulation demonstrations across many scenes, tasks, collectors, and institutions, documenting the practical cost of coordinated physical data collection.[29] Such datasets improve environmental diversity, but coverage is still shaped by hardware, operator behavior, task design, and successful collection. The absence of a failure type from a dataset does not make the resulting policy safe against it.

[Open X-Embodiment](https://aiwiki.ai/wiki/open_x_embodiment) supports several open generalist policies. Octo was designed to accept varied observations, goals, and action spaces and to be fine-tuned to new robot setups.[30] [OpenVLA](https://aiwiki.ai/wiki/openvla) connected an open vision-language backbone with robot-action training and released fine-tuning resources.[31] These projects improve inspectability and reuse, but openness of weights does not supply missing data, hardware calibration, or a safety case.

[Transfer Learning](https://aiwiki.ai/wiki/transfer_learning) across embodiments is difficult because a body's actions and reachable states differ. Common approaches map actions to a shared end-effector representation, learn embodiment-specific adapters, condition the policy on robot state, or separate task-level intent from inverse kinematics and control. A representation that is invariant to robot geometry may help semantic transfer while discarding information needed for collision avoidance or balance. Transfer therefore needs both task-level and embodiment-level evaluation.

### World knowledge and action knowledge

Image-text and video pretraining can supply object categories, language relations, and broad visual patterns. Robot data supply action consequences, proprioception, contact, and the meaning of the robot's control interface. These knowledge sources are complementary, not substitutable.

The Gemini Robotics technical report described one model that extends a multimodal foundation model with physical actions and another focused on embodied spatial and temporal reasoning.[32] Because the report is produced by the developer and includes proprietary components, its results should be treated as evidence for the reported evaluation, not as an independently reproduced comparison.

World-model pretraining offers another path. Video can expose far more environments than physical robot collection, while a smaller action-conditioned dataset can connect prediction to control. The unresolved question is which visual regularities are causal and executable. Current research tests both latent prediction and generated futures, and 2026 benchmarks show that physical plausibility remains a separate measurement from visual quality.[33][12]

### What generalist means

"Generalist" is relative to an evaluation set. A policy can be general across objects but fixed to one arm; across robots but limited to tabletop manipulation; across instructions but not across camera layouts; or across simulated worlds but not physical hardware. A generality claim should name the axes that changed.

| Axis | Example held constant | Example varied |
| --- | --- | --- |
| Task | One pick-and-place skill | Grasping, opening, folding, or navigation |
| Object | One training object set | Novel instances, categories, materials, or deformable objects |
| Scene | One laboratory table | New layouts, lighting, clutter, buildings, or outdoor settings |
| Embodiment | One robot and gripper | New arms, bases, hands, sensor positions, or control interfaces |
| Instruction | Fixed templates | New wording, goals, constraints, corrections, or demonstrations |
| Dynamics | Calibrated hardware | Payload, friction, wear, delay, damage, or external disturbance |
| Horizon | One short skill | Multi-stage tasks with memory, recovery, and changing goals |
| People | No person or scripted motion | Unmodeled human movement, collaboration, preference, or interruption |

No single aggregate score establishes generality across all rows. A model can improve average performance while becoming worse on rare or safety-critical conditions.

### Research milestones and their boundaries

| Work | Contribution | Boundary of the evidence |
| --- | --- | --- |
| RT-1 | Large transformer policy trained on diverse real-robot demonstrations | One robot fleet and documented task distribution |
| RT-2 | Co-training of vision-language and robot-action data | Selected semantic transfer tests on specified hardware |
| Open X-Embodiment and RT-X | Standardized multi-institution robot datasets and tested cross-robot transfer | Heterogeneous actions still require normalization and embodiment-specific evaluation |
| DROID | Distributed real-world manipulation dataset | Demonstration coverage, not an autonomous deployment study |
| Octo | Open adaptable generalist policy | Evaluated robots, sensors, and fine-tuning protocols |
| OpenVLA | Open VLA and adaptation tools | Benchmark-specific comparisons, not universal model ranking |
| pi0 | VLM backbone with a continuous flow-matching action expert | Reported task and platform distribution |
| Gemini Robotics | Organization-reported VLA and embodied-reasoning models | Proprietary training and evaluation limit independent reproduction |

The table is selective. It describes ideas that changed research practice and omits valuations, market projections, and demonstrations without enough methodological detail.

## Evaluation and measurement

Embodied evaluation is difficult because a score combines the agent, embodiment, environment, task distribution, and measurement code. Reproducible reporting must expose all of them.

### Experimental unit and protocol

An evaluation should specify:

1. the physical robot or simulated embodiment, including sensors, actuators, controller, and compute;
2. the task distribution and which elements are unseen;
3. the initial-state distribution and reset procedure;
4. the observation and action interfaces available to the policy;
5. the success detector and any partial-credit rule;
6. the time, path, collision, force, energy, or intervention budget;
7. the number of independent trials and uncertainty around estimates;
8. whether a person selects checkpoints, retries, moves objects, or stops unsafe behavior;
9. whether privileged simulator state, external cameras, maps, or labels are used;
10. all model adaptation performed on the evaluation robot or environment.

A trial is not independent if repeated attempts share a changing scene, warm-started state, or operator decisions. Small sample counts make success percentages unstable. Reporting a denominator and confidence interval is more informative than a percentage alone.

### Metrics

Task success is often necessary but insufficient. Navigation may report path efficiency, collisions, and localization error. Manipulation may report completion, time, force, object damage, precision, and recovery. Human-robot tasks may include workload, interruption, comfort, communication success, and near misses.

The success-weighted-by-path-length metric was proposed to combine navigation success with path efficiency while retaining a clear task definition.[16] It does not capture every concern: an efficient path can still be unsafe, and the shortest path may not be socially appropriate.

Long-horizon tasks expose multiplicative failure. If each of ten required stages succeeds independently with probability 0.9, the full sequence succeeds with probability about 0.35. Real stages are not independent, but the example shows why high short-skill success does not imply reliable long tasks. Evaluation should record where failure occurs and whether the system can recover.

### Benchmarks

Benchmarks make comparisons easier by fixing parts of the protocol:

| Benchmark or platform | Main capability | Important boundary |
| --- | --- | --- |
| Habitat | Navigation and interactive embodied tasks | Simulator, scene dataset, sensors, and task configuration determine the claim |
| AI2-THOR | Indoor navigation and object interaction | Unity-based environment and available actions abstract physical execution |
| Room-to-Room | Vision-and-language navigation | Panoramic graph navigation in scanned buildings |
| RLBench | Multi-task robot manipulation | Simulated robot, task definitions, demonstrations, and success detectors |
| CALVIN | Language-conditioned long-horizon manipulation | Simulated continuous control with a specified sequence protocol |
| BEHAVIOR-1K | Everyday activities with many objects and physical states | Ambitious simulated task suite with difficult transfer and long-horizon requirements |
| EmbodiedBench | Multimodal reasoning, navigation, and manipulation | Model and simulator-specific diagnostic tasks rather than physical deployment |
| PAI-Bench | Physical perception and prediction | Tests physical capability separately from generic multimodal performance |
| RoboWM-Bench | Executability of generated manipulation futures | Focuses on translating generated behavior into robot action and execution |

EmbodiedBench reported that the multimodal models it tested performed much better on high-level tasks than on low-level manipulation, illustrating that verbal or visual reasoning scores do not establish motor competence.[34] PAI-Bench extended physical evaluation in 2026 by separating perception and future prediction across controlled tasks.[33] These results are diagnostic, not timeless rankings of named models.

### Generalization and leakage

[Generalization](https://aiwiki.ai/wiki/generalization) needs an explicit split. Novel instruction wording is different from a novel object, scene, embodiment, action interface, or physical parameter. A benchmark can claim "unseen" while varying only one narrow axis.

Pretrained models create leakage concerns because web images, videos, scene scans, object models, or benchmark text may appear in pretraining data. Exact contamination may be impossible to determine for closed datasets. Strong evaluation therefore uses newly collected environments, controlled perturbations, held-out embodiments, and physical trials where possible.

Repeated use of one benchmark can lead researchers to tune architecture, prompts, checkpoints, or resets to its hidden assumptions. Cross-benchmark and cross-laboratory tests reduce this risk. They also reveal implementation dependencies that are invisible in a simulator leaderboard.

### Reproducibility and system reporting

Weights alone do not reproduce an embodied system. Required artifacts can include calibration, control code, robot description, camera drivers, preprocessing, action scaling, task assets, success detectors, safety limits, and reset scripts. Hardware wear and manufacturing tolerances can change outcomes.

Results should distinguish autonomous time from setup time and report human labor. A policy that needs frequent expert resets may be useful research, but it is not equivalent to unattended operation. Likewise, teleoperated data collection and autonomous evaluation are different phases and should not be combined into one productivity claim.

## Safety and reliability

An embodied agent can cause physical harm. Safety therefore has to be treated as a system property, not as a reward term added after training.

### Sources of risk

Risks arise from:

- perception errors, occlusion, and sensor failure;
- incorrect state estimation or calibration;
- unsafe plans or unexpected tool use;
- unstable learned control;
- distribution shift in objects, people, or dynamics;
- excessive force, speed, temperature, or energy;
- delayed communication or computation;
- software, network, power, or actuator faults;
- ambiguous, malicious, or conflicting instructions;
- incorrect recovery after a partial failure;
- human misunderstanding of the robot's capability or intent.

The same model can have different risk depending on the body. A wrong classification in a virtual environment has different consequences from a heavy mobile manipulator making contact with a person.

### Constraints and runtime safeguards

Safe learning research distinguishes several mechanisms: robust or adaptive control, constrained optimization, reachability analysis, safety filters, control barrier functions, uncertainty-aware planning, backup policies, monitoring, and formal certification where the assumptions permit it.[35] A learned policy's average success rate is not a substitute for a bound on a safety-critical variable.

Common runtime safeguards include joint and workspace limits, force and speed limits, collision monitoring, emergency stops, independent watchdogs, geofencing, human-presence detection, and safe fallback states. Their existence does not prove the full system safe, but omitting them from a report hides which component prevented harm.

The safety layer should be evaluated with the policy, because it can change behavior. A filter may prevent collisions while causing deadlock; a conservative limit may make the task impossible; a fallback controller may be unsafe under a sensor fault it was not designed to handle.

### Uncertainty, abstention, and recovery

Embodied systems need to recognize when they are outside supported conditions. Uncertainty estimates can trigger slower motion, another observation, replanning, a fallback policy, or a request for human assistance. Estimated confidence itself can be miscalibrated under distribution shift, so abstention thresholds need physical validation.

Recovery is different from repetition. Retrying the same failed grasp without diagnosing object movement may worsen the state. A recovery policy needs observations that reveal failure and a model of which corrective actions remain safe. Benchmarks should count repeated attempts and human rescues rather than resetting them out of the result.

### Human control and accountability

Shared autonomy distributes control between a person and a machine. The person may set goals, approve plans, teleoperate difficult segments, or interrupt execution. The interface should make control authority, system state, uncertainty, and stop behavior clear.

Responsibility cannot be assigned from the model architecture alone. Deployment decisions involve the system integrator, hardware, operators, environment owner, maintenance, monitoring, and applicable standards. An experimental result is not a certification or a deployment authorization.

Embodied data can also contain sensitive images, audio, locations, routines, and bystanders. Data collection needs consent, minimization, access controls, retention rules, and documentation of how recordings and demonstrations are reused. These requirements are separate from task success.

## Limits and open research questions

Embodied AI has produced systems that learn useful navigation, manipulation, and control behaviors, but broad physical competence remains unresolved.

### Data coverage

Robot data are expensive because they require hardware, calibration, supervision, resets, and safe environments. Large datasets such as Open X-Embodiment and DROID increase diversity but remain small and uneven compared with internet text or images.[28][29] Data concentrate on available platforms, institutions, successful tasks, and visually observable behavior.

Synthetic data can cover rare conditions and vary parameters, but it inherits simulator assumptions. Human video is abundant but lacks robot actions and differs in morphology and viewpoint. Teleoperation produces aligned actions but can be slow and operator dependent. No source replaces all others.

### Cross-embodiment transfer

Semantic task structure may transfer while control does not. "Open the drawer" can be shared across robots, but reach, grasp, force, and trajectory depend on the body. Cross-embodiment learning needs representations that preserve task meaning and adapters that respect geometry, dynamics, and sensing.

Positive average transfer can hide negative transfer for one robot or skill. Reports should include per-embodiment results and compare against training only on the target robot.

### Long horizons and compounding error

Long-horizon behavior requires maintaining task state, detecting completion, remembering constraints, managing resources, and recovering from partial failure. Hierarchies shorten each decision horizon but depend on accurate interfaces between levels.

Language-guided decomposition helps when task steps are semantically clear. It can fail when the physical scene does not support the proposed step or when an earlier action changes object state. Closed-loop grounding must continue throughout execution, not only when the initial plan is produced.

### Physical understanding

Current models can recognize objects and predict plausible visual events without representing the dynamics needed for action. PAI-Bench and RoboWM-Bench were designed in part to make this distinction measurable.[33][12] Physical understanding is not one scalar ability. It includes geometry, persistence, contact, support, friction, deformation, force, timing, and the consequences of intervention.

A useful [computer vision](https://aiwiki.ai/wiki/computer_vision) representation may be invariant to changes that matter for control. Object identity can survive a small pose change while a grasp cannot. Embodied representations therefore need task-relevant precision as well as semantic abstraction.

### Adaptation without unsafe exploration

Hardware, payloads, surfaces, and environments change. An agent that never adapts may degrade, while unconstrained online exploration can be unsafe. Open questions include how to detect change, choose informative but safe actions, retain prior skills, and verify an updated policy.

The distinction among training, calibration, adaptation, and evaluation should remain visible. Updating on test failures can improve a system, but the later trials no longer measure zero-shot performance.

### Body and controller co-design

Most machine-learning research treats robot hardware as fixed. Embodied-intelligence research also asks how morphology, sensing, and materials should be designed for a task.[10] A compliant mechanism may simplify control but reduce precision; more degrees of freedom may expand capability but increase data and control demands.

Co-design creates a difficult optimization problem because changing the body changes the observations, actions, and policy. It also complicates comparison: a better outcome may come from hardware, learning, or their interaction. Factorial experiments and matched control baselines help separate these effects.

### Scope of current claims

Contemporary position papers argue that interaction-centered learning should play a larger role in AI research.[36] That is a research agenda, not evidence that embodiment automatically produces general intelligence. A body supplies constraints and data; the resulting competence still depends on learning objectives, representation, memory, planning, control, and evaluation.

The durable test is behavioral and protocol specific: what can the agent do, with which body and information, in which environments, how reliably, under what intervention and safety limits, and how does performance change outside the training distribution?

## References
1. Harnad, S. "The Symbol Grounding Problem." *Physica D: Nonlinear Phenomena* 42, 1990, pp. 335-346. https://www.sciencedirect.com/science/article/pii/0167278990900876
2. Savva, M., et al. "Habitat: A Platform for Embodied AI Research." *IEEE/CVF International Conference on Computer Vision*, 2019, pp. 9339-9347. https://openaccess.thecvf.com/content_ICCV_2019/html/Savva_Habitat_A_Platform_for_Embodied_AI_Research_ICCV_2019_paper.html
3. Nilsson, N. J., ed. *Shakey the Robot*. SRI International Technical Note 323, 1984. https://www.sri.com/publication/artificial-intelligence-pubs/shakey-the-robot-pub/
4. Bajcsy, R. "Active Perception." *Proceedings of the IEEE* 76, no. 8, 1988, pp. 996-1005. https://repository.upenn.edu/server/api/core/bitstreams/14686d28-7a8a-4554-beee-991eec61cbdd/content
5. Brooks, R. A. "Intelligence without Representation." *Artificial Intelligence* 47, 1991, pp. 139-159. https://people.csail.mit.edu/brooks/papers/representation.pdf
6. Kober, J., Bagnell, J. A., and Peters, J. "Reinforcement Learning in Robotics: A Survey." *The International Journal of Robotics Research* 32, no. 11, 2013, pp. 1238-1274. https://www.ri.cmu.edu/pub_files/2013/7/Kober_IJRR_2013.pdf
7. Argall, B. D., Chernova, S., Veloso, M., and Browning, B. "A Survey of Robot Learning from Demonstration." *Robotics and Autonomous Systems* 57, no. 5, 2009, pp. 469-483. https://users.eecs.northwestern.edu/~argall/docs/09ras_argall.pdf
8. Levine, S., Finn, C., Darrell, T., and Abbeel, P. "End-to-End Training of Deep Visuomotor Policies." *Journal of Machine Learning Research* 17, no. 39, 2016, pp. 1-40. https://www.jmlr.org/papers/v17/15-522.html
9. Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. "Planning and Acting in Partially Observable Stochastic Domains." *Artificial Intelligence* 101, 1998, pp. 99-134. https://www.sciencedirect.com/science/article/pii/S000437029800023X
10. Howard, D., Eiben, A. E., Kennedy, D. F., Mouret, J.-B., Valencia, P., and Winkler, D. "Evolving Embodied Intelligence from Materials to Machines." *Nature Machine Intelligence* 1, 2019, pp. 12-19. https://www.nature.com/articles/s42256-018-0009-9
11. Shi, L. X., et al. "Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models." *Proceedings of the 42nd International Conference on Machine Learning*, 2025, pp. 54919-54933. https://proceedings.mlr.press/v267/shi25d.html
12. Jiang, F., et al. "RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation." *IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops*, 2026, pp. 4455-4460. https://openaccess.thecvf.com/content/CVPR2026W/GigaBrainChallenge/html/Jiang_RoboWM-Bench_A_Benchmark_for_Evaluating_World_Models_in_Robotic_Manipulation_CVPRW_2026_paper.html
13. Hafner, D., et al. "Mastering Diverse Control Tasks through World Models." *Nature* 640, 2025, pp. 647-653. https://www.nature.com/articles/s41586-025-08744-2
14. Assran, M., et al. "V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning." 2025. https://arxiv.org/abs/2506.09985
15. Anderson, P., et al. "Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments." *IEEE Conference on Computer Vision and Pattern Recognition*, 2018, pp. 3674-3683. https://openaccess.thecvf.com/content_cvpr_2018/html/Anderson_Vision-and-Language_Navigation_Interpreting_CVPR_2018_paper.html
16. Anderson, P., et al. "On Evaluation of Embodied Navigation Agents." 2018. https://arxiv.org/abs/1807.06757
17. James, S., Ma, Z., Arrojo, D. R., and Davison, A. J. "RLBench: The Robot Learning Benchmark and Learning Environment." *IEEE Robotics and Automation Letters* 5, no. 2, 2020, pp. 3019-3026. https://arxiv.org/abs/1909.12271
18. Mees, O., Hermann, L., Rosete-Beas, E., and Burgard, W. "CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks." *IEEE Robotics and Automation Letters* 7, no. 3, 2022, pp. 7327-7334. https://calvin.cs.uni-freiburg.de/
19. Li, C., et al. "BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation." 2024. https://arxiv.org/abs/2403.09227
20. Peng, X. B., Andrychowicz, M., Zaremba, W., and Abbeel, P. "Sim-to-Real Transfer of Robotic Control with Dynamics Randomization." *IEEE International Conference on Robotics and Automation*, 2018. https://xbpeng.github.io/projects/SimToReal/SimToReal_2018.pdf
21. Kolve, E., et al. "AI2-THOR: An Interactive 3D Environment for Visual AI." 2017. https://arxiv.org/abs/1712.05474
22. Todorov, E., Erez, T., and Tassa, Y. "MuJoCo: A Physics Engine for Model-Based Control." *IEEE/RSJ International Conference on Intelligent Robots and Systems*, 2012, pp. 5026-5033. https://homes.cs.washington.edu/~todorov/papers/TodorovIROS12.pdf
23. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World." *IEEE/RSJ International Conference on Intelligent Robots and Systems*, 2017. https://doi.org/10.1109/IROS.2017.8202133
24. OpenAI et al. "Solving Rubik's Cube with a Robot Hand." 2019. https://arxiv.org/abs/1910.07113
25. Brohan, A., et al. "RT-1: Robotics Transformer for Real-World Control at Scale." 2022. https://arxiv.org/abs/2212.06817
26. Zitkovich, B., et al. "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control." *Proceedings of the 7th Conference on Robot Learning*, 2023, pp. 2165-2183. https://proceedings.mlr.press/v229/zitkovich23a.html
27. Black, K., et al. "pi0: A Vision-Language-Action Flow Model for General Robot Control." *Robotics: Science and Systems*, 2025. https://www.roboticsproceedings.org/rss21/p010.html
28. Open X-Embodiment Collaboration. "Open X-Embodiment: Robotic Learning Datasets and RT-X Models." 2023. https://arxiv.org/abs/2310.08864
29. Khazatsky, A., et al. "DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset." *Robotics: Science and Systems*, 2024. https://www.roboticsproceedings.org/rss20/p120.html
30. Ghosh, D., et al. "Octo: An Open-Source Generalist Robot Policy." *Robotics: Science and Systems*, 2024. https://www.roboticsproceedings.org/rss20/p090.html
31. Kim, M. J., et al. "OpenVLA: An Open-Source Vision-Language-Action Model." *Proceedings of the 8th Conference on Robot Learning*, 2025, pp. 2679-2713. https://proceedings.mlr.press/v270/kim25c.html
32. Gemini Robotics Team. "Gemini Robotics: Bringing AI into the Physical World." 2025. https://arxiv.org/abs/2503.20020
33. Zhou, F., Huang, J., Li, J., Ramanan, D., and Shi, H. "PAI-Bench: A Comprehensive Benchmark for Physical AI." *IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2026, pp. 21522-21536. https://openaccess.thecvf.com/content/CVPR2026/html/Zhou_PAI-Bench_A_Comprehensive_Benchmark_For_Physical_AI_CVPR_2026_paper.html
34. Yang, R., et al. "EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents." *Proceedings of the 42nd International Conference on Machine Learning*, 2025, pp. 70576-70631. https://proceedings.mlr.press/v267/yang25f.html
35. Brunke, L., et al. "Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning." *Annual Review of Control, Robotics, and Autonomous Systems* 5, 2022, pp. 411-444. https://www.annualreviews.org/content/journals/10.1146/annurev-control-042920-020211
36. Paolo, G., Gonzalez-Billandon, J., and Kegl, B. "Position: A Call for Embodied AI." *Proceedings of the 41st International Conference on Machine Learning*, 2024, pp. 39493-39508. https://proceedings.mlr.press/v235/paolo24a.html