FOMO-Dex

FOMO-Dex (short for Force-Modulated Dexterity) is a robot control method for contact-rich dexterous manipulation. Researchers at Amazon Frontier AI & Robotics (FAR) and Carnegie Mellon University introduced it in an arXiv preprint submitted on 12 October 2026.^[1][3] The method treats an arm-and-hand motion command, coming from a human teleoperator or a learned policy, as a "nominal action". It then uses force feedback to adjust only the parts of that command that matter for the current contact, and leaves the other parts unchanged.^[1] The authors tested it on four tasks: picking up a poker card from a flat surface, cutting paper with a utility knife, turning a nut along a bolt, and driving a screw with a screwdriver.^[1] They report that it reduced excessive contact forces compared with a stiff controller, that operators in a four-person user study collected more successful demonstration data per hour once they had practiced, and that policies trained on demonstrations collected with it outperformed policies trained on stiff-control demonstrations on the rotation tasks.^[1]

Background

Collecting demonstrations by teleoperation is a common way to get training data for robot learning. The FOMO-Dex paper argues that contact-rich dexterous tasks are hard to demonstrate this way. An operator can specify the motion, but without feeling the robot's contacts they cannot easily reproduce the small force adjustments people make without thinking. During contact, small position errors can then produce large and inconsistent forces.^[1]

Compliance, meaning a controlled give in response to contact force, is a long-standing answer to this kind of uncertainty. Marc Raibert and John Craig's hybrid position/force control (1981) separates directions governed by motion from directions governed by force.^[9] Neville Hogan's impedance control, presented at the 1984 American Control Conference and published in three parts in the Journal of Dynamic Systems, Measurement, and Control in 1985, specifies the dynamic relationship between a manipulator's motion and its interaction forces.^[7][8] In admittance control, the form FOMO-Dex uses, a measured force error is converted into a motion correction. The "task frame formalism" expresses motion and force constraints in coordinates aligned with the task geometry.^[1] The FOMO-Dex authors say their work builds on these task-frame and admittance principles "rather than introducing a new admittance law".^[1]

The paper's point is that making a multi-fingered hand uniformly softer is not enough. A hand can make and break several contacts at once, and those contacts are coupled through the object. With a tool, the robot should keep moving along the task direction while yielding in constrained directions. When holding a grasp, it should regulate fingertip forces without suppressing the motion that turns the object.^[1]

Authors

Affiliations are as printed on the paper (1 = Amazon FAR, 2 = Carnegie Mellon University).^[1] The project page gives the same two affiliations.^[2]

System setup

Source for all rows: the FOMO-Dex paper.^[1] The controller runs at 90 Hz in all tasks, and trials were recorded at 30 Hz, logging operator commands, post-compliance commands, robot state, wrist wrench and fingertip forces.^[1]

Method

Nominal action plus correction

At each time step the nominal command is a Cartesian end-effector pose plus a vector of finger-joint targets. The controller computes a force-derived correction and combines it with the nominal command: addition for finger joints, and pose composition for the arm.^[1] Because the method works on motion commands, the nominal action can come from either a teleoperator or a learned policy.^[1]

The correction is built in three steps:^[1]

1. Choose interaction directions. From the task geometry and the current contacts, the controller defines a set of unit directions. Each direction is paired with a force measurement from the relevant sensor (wrist force-torque, fingertip tactile, or joint torque), and the force is projected onto it. Only a selected subset of directions is modulated; directions meant to stay stiff get no correction.
2. Compute a force error and pass it through admittance. If the goal is to track a target force, the error is the signed difference from that target. If the goal is only to cap loading, the error is one-sided: zero below a force limit and positive above it. A direction-wise admittance law, m·ξ'' + d·ξ' + k·ξ = e, with virtual mass, damping and stiffness for each direction, turns the error into a scalar correction. Setting the mass to zero gives a first-order response, and setting mass and stiffness to zero gives an integrating response. The state follows its contact as the direction rotates, and resets when a contact is made or lost or when a direction enters or leaves the modulated set.
3. Map the correction to the robot. Arm corrections are summed and used to shift the equilibrium pose given to the arm's impedance controller. Finger corrections become desired fingertip displacements, which are converted to joint corrections with damped least-squares inverse kinematics.

Three task-specific realizations

Source: the FOMO-Dex paper.^[1] The project page also shows a chip pickup as a second example of unilateral load relief.^[2]

Controller settings

Source: Table 1 of the paper.^[1]

Experiments

All results below are reported by the authors in the preprint, which had not been peer reviewed as of 1 November 2026.^[1]

Teleoperation with three controllers

The paper compares FOMO-Dex with two baselines that share the same teleoperation interface, command rate, sensing, low-level control and safety limits. The stiff controller executes the operator command directly (hand position control and arm impedance control, no correction). The naive compliant controller uses the same admittance parameters but applies them in all directions instead of selectively along task- and contact-relevant directions.^[1]

Source: Table 2 of the paper, which the caption describes as covering 50 trials. "Force exceedance" is the time-averaged amount by which the measured contact force exceeds a task-specific safe threshold.^[1]

The authors do not claim FOMO-Dex wins every metric. In bolt-nut turning, stiff control completed more turns at the cost of larger contact forces, naive compliance cut forces but suppressed rotation more, and FOMO-Dex fell between the two on both measures. In screwdriving, naive compliance had slightly lower force exceedance, but completed fewer turns and slipped more. The paper concludes that "lower force exceedance alone therefore does not imply better manipulation performance".^[1] For card picking, naive compliance reduced force but succeeded in only 37.5% of trials, which the authors attribute to relaxing the commanded motion indiscriminately.^[1]

User study

Four participants teleoperated card picking, bolt-nut turning and screwdriving with both FOMO-Dex and the stiff controller, in randomized order. Each did 10 trials per condition with no practice, then had 10 minutes of practice with both controllers, then did another 10 trials per controller. The main measure was successful demonstration time per hour of teleoperation.^[1]

Source: Table 3 of the paper.^[1] The two controllers were about equal on bolt-nut turning without practice; the paper reports higher throughput with FOMO-Dex on all three tasks after practice.^[1] The study had four participants, and several cells have large standard deviations.

Policies trained on the demonstrations

The authors trained visuomotor policies by behavioral cloning. The policy takes RGB images from the head camera, robot proprioception and force measurements. A frozen pretrained DINOv2 encoder extracts image features and a diffusion policy predicts joint-space actions. Proprioceptive inputs are randomly masked during training to push the policy toward visual and force cues.^[1] Because each teleoperated episode records both the operator's raw command and the force-modulated command actually sent to the robot, the authors could compare which of the two the policy should imitate.^[1]

Source: Table 4 of the paper; each task-controller setting used 50 training demonstrations.^[1] Column labels in the first table follow the paper. The policy trained on raw commands completed no successful lifts even with force modulation applied at deployment, so the authors used the post-modulation actions as training targets for the other tasks.^[1]

Relation to prior work

The paper places FOMO-Dex among several lines of work:^[1]

• Dexterous teleoperation. Recent systems map tracked human motion to high-DoF robot hands, and the paper cites DexPilot and AnyTeleop among the teleoperation systems used to collect demonstrations. Bunny-VisionPro and DOGlove add haptic feedback to the operator. DexUMI uses a wearable hand exoskeleton. FOMO-Dex instead puts the force response on the robot side, so the operator does not have to make every correction.
• Object-level impedance control for hands. Earlier methods by Li et al. (2014) and Pfanne et al. (2020) control a desired object motion through the fingers. FOMO-Dex corrects nominal arm and finger commands and does not need an object-pose reference.
• Learning compliance. Adaptive Compliance Policy, UMI-FT and VIDP learn stiffness or compliance parameters, and Force Policy learns an interaction frame for hybrid control. In FOMO-Dex the task configuration specifies the force-response directions and feedback laws, which act on nominal commands without a learned compliance or residual model.
• Force-informed action targets. DexForce reconstructs fingertip position targets from kinesthetic demonstrations, and Contact-Grounded Policy learns a mapping from predicted states to controller targets. FOMO-Dex records the corrected commands directly during collection and imitates them.

The reference list also includes earlier dexterity work co-authored by Qi and Malik, including DexScrew (ICRA 2026), the OSMO tactile glove (RA-L 2026) and ForceBand (CoRL 2026).^[1]

Limitations and future work

In the current method the interaction directions and force limits are configured per task (Table 1); in the rotation tasks the per-contact force targets are derived from a slowly adapted grip level within a configured 0.5-2.4 N range.^[1] The paper names three directions for future work: inferring interaction directions and force targets online, adding uncertainty-aware contact estimation, and evaluating the framework on a wider range of tasks and objects.^[1] All experiments use one arm-hand platform.^[1]

Availability

The preprint is on arXiv as 2610.17161 (categories cs.RO and cs.AI), and the project page hosts videos of teleoperated and autonomous rollouts.^[1][2] As of 1 November 2026, the project page listed the code as "Coming Soon".^[2]

Continue reading

• AmazonAmazon.com, Inc. is an American public company whose businesses include online and physical stores, third-party seller services, Amazon Web Services (AWS), advertising, subscriptions, devices, logistics, and…Mentioned 7 times above
• TTeleoperationTeleoperation is the operation of a machine by a human at a distance, with commands carried over a communication link and the machine's cameras, microphones and force sensors standing in for the operator's own…Mentioned 6 times above
• TTactile sensingTactile sensing is the technology that gives robots a sense of touch: the ability to measure the physical contact between a robot and the world, including contact force, pressure distribution, shear, slip…Mentioned 3 times above
• SharpaSharpa is a Singapore-headquartered artificial intelligence robotics company, founded at the end of 2024, that builds general-purpose robots and dexterous robotic hands with combined vision and touch sensing.Mentioned 3 times above
• DDexterous handA dexterous hand (also called a dexterous robotic hand or multi-fingered robot hand) is an anthropomorphic end effector that reproduces the grasping, in-hand manipulation, and tactile perception of the human…Listed in See also
• IImitation LearningImitation learning is a family of methods for learning sequential behavior from demonstrations.Listed in See also

More to read

• Carnegie Mellon University
• Robot manipulation
• Behavioral Cloning
• Robot learning
• Diffusion policy
• Force-torque sensor

Topics:AI ResearchEmbodied AIMachine LearningRobotics

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34Ying Yuan, David Held, Jitendra Malik, Haozhi Qi. "Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation." arXiv:2610.17161, 12 October 2026. arxiv.org/...2610.17161
2. ^1 ^2 ^3 ^4"FOMO-DEX." Project page for Force-Modulated Dexterity. yingyuan0414.github.io/fomo-dex (accessed 1 November 2026)
3. ^1 ^2Haozhi Qi. Personal homepage. haozhi.io (accessed 1 November 2026)
4. ^Ying Yuan. Personal homepage. yingyuan0414.github.io (accessed 1 November 2026)
5. ^David Held. Personal homepage. davheld.github.io (accessed 1 November 2026)
6. ^Jitendra Malik. Home page, University of California, Berkeley. people.eecs.berkeley.edu/~malik (accessed 1 November 2026)
7. ^Neville Hogan. "Impedance Control: An Approach to Manipulation." 1984 American Control Conference. doi.org/...ACC.1984.4788393
8. ^Neville Hogan. "Impedance Control: An Approach to Manipulation: Part I, Theory." *Journal of Dynamic Systems, Measurement, and Control*, March 1985. doi.org/...1.3140702
9. ^Marc H. Raibert and John J. Craig. "Hybrid Position/Force Control of Manipulators." *Journal of Dynamic Systems, Measurement, and Control*, 1981. doi.org/...1.3139652

Show all 9 references