# FOMO-Dex

> Source: https://aiwiki.ai/wiki/fomo_dex
> Updated: 2026-11-01
> Fact-checked: 2026-11-01
> Categories: AI Research, Embodied AI, Machine Learning, Robotics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "FOMO-Dex." aiwiki.ai, 1 Nov 2026. https://aiwiki.ai/wiki/fomo_dex
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**FOMO-Dex** (short for **Force-Modulated Dexterity**) is a robot control method for contact-rich [dexterous manipulation](https://aiwiki.ai/wiki/robot_manipulation). Researchers at Amazon Frontier AI & Robotics (FAR) and [Carnegie Mellon University](https://aiwiki.ai/wiki/cmu) introduced it in an arXiv preprint submitted on 12 October 2026.[1][3] The method treats an arm-and-hand motion command, coming from a human teleoperator or a learned policy, as a "nominal action". It then uses force feedback to adjust only the parts of that command that matter for the current contact, and leaves the other parts unchanged.[1] The authors tested it on four tasks: picking up a poker card from a flat surface, cutting paper with a utility knife, turning a nut along a bolt, and driving a screw with a screwdriver.[1] They report that it reduced excessive contact forces compared with a stiff controller, that operators in a four-person user study collected more successful demonstration data per hour once they had practiced, and that policies trained on demonstrations collected with it outperformed policies trained on stiff-control demonstrations on the rotation tasks.[1]

## Background

Collecting demonstrations by [teleoperation](https://aiwiki.ai/wiki/teleoperation) is a common way to get training data for [robot learning](https://aiwiki.ai/wiki/robot_learning). The FOMO-Dex paper argues that contact-rich dexterous tasks are hard to demonstrate this way. An operator can specify the motion, but without feeling the robot's contacts they cannot easily reproduce the small force adjustments people make without thinking. During contact, small position errors can then produce large and inconsistent forces.[1]

Compliance, meaning a controlled give in response to contact force, is a long-standing answer to this kind of uncertainty. Marc Raibert and John Craig's hybrid position/force control (1981) separates directions governed by motion from directions governed by force.[9] Neville Hogan's impedance control, presented at the 1984 American Control Conference and published in three parts in the *Journal of Dynamic Systems, Measurement, and Control* in 1985, specifies the dynamic relationship between a manipulator's motion and its interaction forces.[7][8] In admittance control, the form FOMO-Dex uses, a measured force error is converted into a motion correction. The "task frame formalism" expresses motion and force constraints in coordinates aligned with the task geometry.[1] The FOMO-Dex authors say their work builds on these task-frame and admittance principles "rather than introducing a new admittance law".[1]

The paper's point is that making a multi-fingered hand uniformly softer is not enough. A hand can make and break several contacts at once, and those contacts are coupled through the object. With a tool, the robot should keep moving along the task direction while yielding in constrained directions. When holding a grasp, it should regulate fingertip forces without suppressing the motion that turns the object.[1]

## Authors

| Author | Affiliation listed in the paper | Background |
|---|---|---|
| Ying Yuan | Amazon FAR; Carnegie Mellon University | PhD student at the CMU Robotics Institute advised by David Held; her homepage says she started an internship at Amazon FAR in June 2026[4] |
| David Held | Carnegie Mellon University | Associate Professor at the CMU Robotics Institute and director of the Robots Perceiving and Doing (RPAD) lab[5] |
| Jitendra Malik | Amazon FAR | Arthur J. Chick Professor of EECS at the [University of California, Berkeley](https://aiwiki.ai/wiki/uc_berkeley)[6] |
| Haozhi Qi | Amazon FAR | Member of Technical Staff at Amazon FAR and incoming Assistant Professor of computer science at the University of Chicago; PhD from UC Berkeley, advised by Yi Ma and Jitendra Malik[3] |

Affiliations are as printed on the paper (1 = Amazon FAR, 2 = Carnegie Mellon University).[1] The project page gives the same two affiliations.[2]

## System setup

| Component | Hardware |
|---|---|
| Arm | 7-DoF Tianji Marvin arm |
| Hand | 22-DoF Sharpa Wave dexterous hand, made by [Sharpa](https://aiwiki.ai/wiki/sharpa) |
| Arm-level sensing | Six-axis [force-torque sensor](https://aiwiki.ai/wiki/force_torque_sensor) at the wrist |
| Finger-level sensing | The hand's fingertip [tactile](https://aiwiki.ai/wiki/tactile_sensing) and joint-torque measurements |
| Operator finger tracking | Sharpa Avatar glove |
| Operator hand-pose tracking | HTC Vive Tracker attached to the glove |
| Policy camera | Head-mounted ZED camera (RGB) |

Source for all rows: the FOMO-Dex paper.[1] The controller runs at 90 Hz in all tasks, and trials were recorded at 30 Hz, logging operator commands, post-compliance commands, robot state, wrist wrench and fingertip forces.[1]

## Method

### Nominal action plus correction

At each time step the nominal command is a Cartesian end-effector pose plus a vector of finger-joint targets. The controller computes a force-derived correction and combines it with the nominal command: addition for finger joints, and pose composition for the arm.[1] Because the method works on motion commands, the nominal action can come from either a teleoperator or a learned policy.[1]

The correction is built in three steps:[1]

1. **Choose interaction directions.** From the task geometry and the current contacts, the controller defines a set of unit directions. Each direction is paired with a force measurement from the relevant sensor (wrist force-torque, fingertip tactile, or joint torque), and the force is projected onto it. Only a selected subset of directions is modulated; directions meant to stay stiff get no correction.
2. **Compute a force error and pass it through admittance.** If the goal is to track a target force, the error is the signed difference from that target. If the goal is only to cap loading, the error is one-sided: zero below a force limit and positive above it. A direction-wise admittance law, m·ξ'' + d·ξ' + k·ξ = e, with virtual mass, damping and stiffness for each direction, turns the error into a scalar correction. Setting the mass to zero gives a first-order response, and setting mass and stiffness to zero gives an integrating response. The state follows its contact as the direction rotates, and resets when a contact is made or lost or when a direction enters or leaves the modulated set.
3. **Map the correction to the robot.** Arm corrections are summed and used to shift the equilibrium pose given to the arm's impedance controller. Finger corrections become desired fingertip displacements, which are converted to joint corrections with damped least-squares inverse kinematics.

### Three task-specific realizations

| Realization | Task | What is modulated | What stays unchanged |
|---|---|---|---|
| Unilateral load relief | Card picking | A direction pointing away from the support surface; above the load limit the contacting finger retreats through its joints | Motion below the limit; no fixed contact force is enforced |
| Translational tool compliance | Paper cutting with a utility knife | The arm yields to lateral forces from either side, and retreats along the surface normal when pressing force exceeds its limit | Motion along the cutting direction |
| Multi-contact force regulation | Bolt-nut turning, screwdriving | Fingertip forces along local contact normals, driven toward compressive targets derived from a slowly adapted shared grip level | The commanded rotational motion |

Source: the FOMO-Dex paper.[1] The project page also shows a chip pickup as a second example of unilateral load relief.[2]

### Controller settings

| Setting | Card picking | Paper cutting | Bolt-nut / screwdriver |
|---|---|---|---|
| Controller rate | 90 Hz | 90 Hz | 90 Hz |
| Force target | Not applicable | Not applicable | 0.5-2.4 N |
| Force limit | 3.0 N | 3.5 N | 3.0 N |
| Contact on/off thresholds | 0.6 / 0.6 N | 5.0 / 2.0 N | 0.25 / 0.15 N |
| Maximum correction | 0.05 rad | 0.06 m | 0.10 rad |

Source: Table 1 of the paper.[1]

## Experiments

All results below are reported by the authors in the preprint, which had not been peer reviewed as of 1 November 2026.[1]

### Teleoperation with three controllers

The paper compares FOMO-Dex with two baselines that share the same teleoperation interface, command rate, sensing, low-level control and safety limits. The **stiff** controller executes the operator command directly (hand position control and arm impedance control, no correction). The **naive compliant** controller uses the same admittance parameters but applies them in all directions instead of selectively along task- and contact-relevant directions.[1]

| Task | Metric | Stiff | Naive compliant | FOMO-Dex |
|---|---|---|---|---|
| Card picking | Success (%), higher is better | 90.9 | 37.5 | 93.8 |
| Card picking | Duration (s), lower is better | 22.1 ± 4.2 | 17.2 ± 2.4 | 16.9 ± 4.2 |
| Card picking | Frames over force threshold (%), lower is better | 13.0 ± 5.4 | 7.4 ± 7.9 | 5.9 ± 2.8 |
| Card picking | Force exceedance (N), lower is better | 0.32 ± 0.29 | 0.10 ± 0.13 | 0.05 ± 0.03 |
| Paper cutting | Cut rate (cm/s), higher is better | 2.47 ± 0.15 | 3.16 ± 1.12 | 2.63 ± 0.41 |
| Paper cutting | Lateral force (N), lower is better | 2.00 ± 0.41 | 1.61 ± 0.43 | 1.41 ± 0.14 |
| Paper cutting | Normal force (N), lower is better | 4.39 ± 0.94 | 5.28 ± 0.77 | 3.62 ± 0.18 |
| Bolt-nut | Turns per episode, higher is better | 23.7 ± 1.2 | 17.0 ± 2.1 | 20.3 ± 1.2 |
| Bolt-nut | Frames over force threshold (%), lower is better | 27.4 ± 4.1 | 8.0 ± 2.0 | 23.3 ± 4.3 |
| Bolt-nut | Force exceedance (N), lower is better | 0.64 ± 0.18 | 0.09 ± 0.03 | 0.57 ± 0.21 |
| Screwdriving | Turns per episode, higher is better | 4.2 ± 2.2 | 14.6 ± 6.9 | 34.7 ± 5.7 |
| Screwdriving | Slip events per episode, lower is better | 1.00 ± 0.00 | 0.75 ± 0.43 | 0.28 ± 0.46 |
| Screwdriving | Frames over force threshold (%), lower is better | 0.4 ± 0.9 | 0.2 ± 0.2 | 0.3 ± 1.2 |
| Screwdriving | Force exceedance (N), lower is better | 0.186 ± 0.073 | 0.001 ± 0.001 | 0.002 ± 0.009 |

Source: Table 2 of the paper, which the caption describes as covering 50 trials. "Force exceedance" is the time-averaged amount by which the measured contact force exceeds a task-specific safe threshold.[1]

The authors do not claim FOMO-Dex wins every metric. In bolt-nut turning, stiff control completed more turns at the cost of larger contact forces, naive compliance cut forces but suppressed rotation more, and FOMO-Dex fell between the two on both measures. In screwdriving, naive compliance had slightly lower force exceedance, but completed fewer turns and slipped more. The paper concludes that "lower force exceedance alone therefore does not imply better manipulation performance".[1] For card picking, naive compliance reduced force but succeeded in only 37.5% of trials, which the authors attribute to relaxing the commanded motion indiscriminately.[1]

### User study

Four participants teleoperated card picking, bolt-nut turning and screwdriving with both FOMO-Dex and the stiff controller, in randomized order. Each did 10 trials per condition with no practice, then had 10 minutes of practice with both controllers, then did another 10 trials per controller. The main measure was successful demonstration time per hour of teleoperation.[1]

| Task | Experience | Stiff (min/h) | FOMO-Dex (min/h) |
|---|---|---|---|
| Card picking | Zero-shot | 11.9 ± 5.2 | 20.6 ± 4.0 |
| Card picking | After 10 min practice | 24.3 ± 9.4 | 29.3 ± 5.6 |
| Bolt-nut | Zero-shot | 41.0 ± 1.1 | 40.7 ± 1.0 |
| Bolt-nut | After 10 min practice | 30.3 ± 12.8 | 50.2 ± 7.0 |
| Screwdriving | Zero-shot | 11.8 ± 6.4 | 17.9 ± 15.2 |
| Screwdriving | After 10 min practice | 12.4 ± 4.7 | 30.7 ± 10.9 |

Source: Table 3 of the paper.[1] The two controllers were about equal on bolt-nut turning without practice; the paper reports higher throughput with FOMO-Dex on all three tasks after practice.[1] The study had four participants, and several cells have large standard deviations.

### Policies trained on the demonstrations

The authors trained visuomotor policies by [behavioral cloning](https://aiwiki.ai/wiki/behavioral_cloning). The policy takes RGB images from the head camera, robot proprioception and force measurements. A frozen pretrained [DINOv2](https://aiwiki.ai/wiki/dinov2) encoder extracts image features and a [diffusion policy](https://aiwiki.ai/wiki/diffusion_policy) predicts joint-space actions. Proprioceptive inputs are randomly masked during training to push the policy toward visual and force cues.[1] Because each teleoperated episode records both the operator's raw command and the force-modulated command actually sent to the robot, the authors could compare which of the two the policy should imitate.[1]

| Card-picking variant (18 evaluation trials) | Reach (%) | Retr. (%) | Lift (%) |
|---|---|---|---|
| Stiff | 100 | 100 | 56 |
| Comp. Raw: train on raw commands, apply FOMO-Dex at inference | 100 | 89 | 0 |
| Comp. Applied: train on force-modulated commands, stiff controller at inference | 100 | 89 | 67 |

| Task (10 evaluation trials) | Policy from stiff demonstrations (turns/ep.) | Policy from FOMO-Dex demonstrations (turns/ep.) |
|---|---|---|
| Bolt-nut | 24.5 ± 4.2 | 60.8 ± 4.5 |
| Screwdriving | 0.6 ± 0.8 | 6.3 ± 2.2 |

Source: Table 4 of the paper; each task-controller setting used 50 training demonstrations.[1] Column labels in the first table follow the paper. The policy trained on raw commands completed no successful lifts even with force modulation applied at deployment, so the authors used the post-modulation actions as training targets for the other tasks.[1]

## Relation to prior work

The paper places FOMO-Dex among several lines of work:[1]

- **Dexterous teleoperation.** Recent systems map tracked human motion to high-DoF robot hands, and the paper cites DexPilot and AnyTeleop among the teleoperation systems used to collect demonstrations. Bunny-VisionPro and DOGlove add haptic feedback to the operator. DexUMI uses a wearable hand exoskeleton. FOMO-Dex instead puts the force response on the robot side, so the operator does not have to make every correction.
- **Object-level impedance control for hands.** Earlier methods by Li et al. (2014) and Pfanne et al. (2020) control a desired object motion through the fingers. FOMO-Dex corrects nominal arm and finger commands and does not need an object-pose reference.
- **Learning compliance.** Adaptive Compliance Policy, UMI-FT and VIDP learn stiffness or compliance parameters, and Force Policy learns an interaction frame for hybrid control. In FOMO-Dex the task configuration specifies the force-response directions and feedback laws, which act on nominal commands without a learned compliance or residual model.
- **Force-informed action targets.** DexForce reconstructs fingertip position targets from kinesthetic demonstrations, and Contact-Grounded Policy learns a mapping from predicted states to controller targets. FOMO-Dex records the corrected commands directly during collection and imitates them.

The reference list also includes earlier dexterity work co-authored by Qi and Malik, including DexScrew (ICRA 2026), the OSMO tactile glove (RA-L 2026) and ForceBand (CoRL 2026).[1]

## Limitations and future work

In the current method the interaction directions and force limits are configured per task (Table 1); in the rotation tasks the per-contact force targets are derived from a slowly adapted grip level within a configured 0.5-2.4 N range.[1] The paper names three directions for future work: inferring interaction directions and force targets online, adding uncertainty-aware contact estimation, and evaluating the framework on a wider range of tasks and objects.[1] All experiments use one arm-hand platform.[1]

## Availability

The preprint is on arXiv as 2610.17161 (categories cs.RO and cs.AI), and the project page hosts videos of teleoperated and autonomous rollouts.[1][2] As of 1 November 2026, the project page listed the code as "Coming Soon".[2]

## See also

- [Dexterous hand](https://aiwiki.ai/wiki/dexterous_hand)
- [Teleoperation](https://aiwiki.ai/wiki/teleoperation)
- [Imitation learning](https://aiwiki.ai/wiki/imitation_learning)
- [Tactile sensing](https://aiwiki.ai/wiki/tactile_sensing)
- [Sharpa](https://aiwiki.ai/wiki/sharpa)
- [Amazon](https://aiwiki.ai/wiki/amazon)

## References

1. Ying Yuan, David Held, Jitendra Malik, Haozhi Qi. "Force-Modulated Dexterity: Unified Compliance for Contact-Rich Manipulation." arXiv:2610.17161, 12 October 2026. https://arxiv.org/abs/2610.17161
2. "FOMO-DEX." Project page for Force-Modulated Dexterity. https://yingyuan0414.github.io/fomo-dex/ (accessed 1 November 2026)
3. Haozhi Qi. Personal homepage. https://haozhi.io/ (accessed 1 November 2026)
4. Ying Yuan. Personal homepage. https://yingyuan0414.github.io/ (accessed 1 November 2026)
5. David Held. Personal homepage. https://davheld.github.io/ (accessed 1 November 2026)
6. Jitendra Malik. Home page, University of California, Berkeley. https://people.eecs.berkeley.edu/~malik/ (accessed 1 November 2026)
7. Neville Hogan. "Impedance Control: An Approach to Manipulation." 1984 American Control Conference. https://doi.org/10.23919/ACC.1984.4788393
8. Neville Hogan. "Impedance Control: An Approach to Manipulation: Part I, Theory." *Journal of Dynamic Systems, Measurement, and Control*, March 1985. https://doi.org/10.1115/1.3140702
9. Marc H. Raibert and John J. Craig. "Hybrid Position/Force Control of Manipulators." *Journal of Dynamic Systems, Measurement, and Control*, 1981. https://doi.org/10.1115/1.3139652