# Lumo-2

> Source: https://aiwiki.ai/wiki/lumo_2
> Updated: 2026-07-25
> Categories: AI Models, Embodied AI, Physical AI, Robotics, World Models
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution to "AI Wiki (aiwiki.ai)".

Lumo-2 is a 4B-scale robot-learning model developed by [Astribot](/wiki/astribot). Astribot describes it as a latent world-action model: instead of rendering a future video before acting, it predicts an action-relevant representation of future change in latent space and uses that representation to generate robot action chunks. The model accepts visual observations, a natural-language instruction, robot state, and recent action history. Astribot's technical report was submitted to arXiv on July 13, 2026, followed by a public project page and task videos.[1][2][17][18]

The model was built by continuing training from [Qwen3.5](/wiki/qwen3_5)-4B and adding learned representations for latent world dynamics and robot actions.[1][15] Astribot evaluated it on the company's [Astribot S1](/wiki/astribot_s1) bimanual mobile manipulator.[1][16] The paper reports results on embodied vision-language benchmarks, generalizable pick-and-place tests, two world-dynamics ablations, human-to-robot adaptation, and 22 controlled real-robot tasks.[1]

| Field | Detail |
| --- | --- |
| Developer | Astribot Team |
| First technical report | July 13, 2026 |
| Model class | Latent world-action model |
| Base model | Qwen3.5-4B |
| Main robot used for evaluation | Astribot S1 |
| Inputs | Natural-language instruction, multi-view visual observations, proprioceptive state, recent action history |
| Learned outputs | Latent world dynamics, discrete action tokens and continuous action chunks, vision-language text outputs |
| Real-robot evaluation | 22 manipulation tasks across six capability categories |
| Public artifacts in the July 2026 release | Project page, report, and demonstration videos; no Lumo-2 weights or training code linked |

## Background

Astribot's earlier Lumo-1 model used explicit structured text for planning before action generation. Its training combined embodied vision-language data, cross-embodiment robot data, reasoning-action trajectories, and reinforcement learning.[3] Lumo-2 replaces the explicit text-planning path with a compact latent representation intended to encode the physical changes relevant to the next action. This places it between a [vision-language-action model](/wiki/vision_language_action_model), which maps observations and instructions to actions, and a video-based [world model](/wiki/world_model), which may generate predicted future observations.[1]

In the Lumo-2 formulation, a high-level distribution predicts latent world dynamics from the current observation, language instruction, and historical context. A low-level distribution then generates an action chunk conditioned on the current observation and that latent representation. Both distributions are parameterized within one multimodal transformer. The latent representation is not a rendered future frame or a general physics simulator. It is a learned feature-space summary trained to retain visual changes that help predict control signals.[1]

The paper calls this process latent reasoning. That label refers to an internal representation learned from visual transitions and actions, not a verbal chain of thought. During inference, the model does not produce an explanation of its plan before acting.[1]

## Architecture

### Latent world dynamics and action tokens

The first training stage uses a dual-branch vector-quantized architecture. Its vision branch compares visual features from a current observation and a later observation. Its action branch encodes the action sequence connecting those observations. A cross-modal fusion module makes the visual representation contribute to action reconstruction, while the action objective pressures the visual representation to focus on motion-related changes rather than background or illumination changes.[1]

The vision branch uses a frozen [DINOv2](/wiki/dinov2) feature extractor.[4] Each input contains five 224 by 224 frames sampled every eight frames from 30 FPS video. Robot examples combine a head-camera view with wrist-camera views through cross-attention; egocentric human video uses the head view alone. The action branch separates torso and two-arm controls into eight groups covering translation, rotation, and gripper motion. It compresses 32 action timesteps into four temporal positions before mapping them to a vector-quantized codebook.[1]

### Historical context

Lumo-2 keeps a fixed-length queue of recent robot actions. A semantic action encoder converts the history to compact tokens, which are concatenated with the current observation, instruction, and state. The paper uses this mechanism to address perceptual aliasing, where two images look similar even though the task is at a different phase. Pouring transparent water is one example: the current frame alone may not show whether pouring should continue or has already finished.[1]

For long-horizon task post-training, Astribot also used a visual history trace. The system projected the two end-effector trajectories onto the main camera image and encoded time through a color gradient. This was an additional task-training mechanism, not evidence of persistent memory across sessions.[1]

### Block-wise action decoding

Standard autoregressive decoding produces one action token at a time. Lumo-2 groups weakly coupled action dimensions into semantic blocks and predicts a block in one forward pass. The paper calls this block-wise autoregression, or BAR. Its causal attention mask permits attention within each block while maintaining ordering between blocks. In the reported setup, BAR reduced 32 sequential action-generation steps to four block-wise steps.[1]

## Three-stage training

| Stage | Objective | Frozen and updated components | Reported training compute |
| --- | --- | --- | --- |
| 1. World dynamics and actions | Learn vector-quantized visual-dynamics and action representations with bidirectional constraints | DINOv2 feature extractor frozen; dynamics, action, codebook, and fusion components trained | 30,000 steps on 64 [NVIDIA H100](/wiki/nvidia_h100) GPUs, per-GPU batch size 24 |
| 2. Vision, language, and action alignment | Add semantic action tokens and cross-modal objectives | Qwen3.5-4B backbone and LLM, Stage 1 encoder, and original codebook frozen; newly introduced semantic, projection, decoding, and loss-specific components updated | 12,000 steps on 64 H100 GPUs |
| 3. Mixed-data co-training | Jointly train vision-language understanding, latent-world prediction, and action generation | Initialized from Stage 2; the paper does not state that the Qwen backbone remains frozen in this stage | 120,000 steps on 160 H100 GPUs |

Stage 2 adds 32 visual-context tokens, 1,024 semantic-action tokens, and one action-sequence placeholder to the vocabulary. Its objectives include action reconstruction, behavior description from vision and action, action prediction from vision and language, cross-modal prediction between actions and world dynamics, and contrastive or discriminative action losses.[1]

Stage 3 mixes three example formats. VLA examples contain vision, language, and action. VLW examples supervise vision, language, and world dynamics. VLWA examples contain all four. Vision-language examples retain standard autoregressive text training, while egocentric video receives latent-world supervision without an action label. Training uses packed sequences up to 8,192 tokens, dynamic visual resolution from 50,176 to 307,200 pixels, and 0.5 dropout on historical-action inputs.[1]

The distinction between stages matters. The supplied technical commentary described Lumo-2 as using a frozen Qwen3.5-4B backbone, which is accurate for Stage 2. The technical report does not say the backbone remains frozen during Stage 3.[1]

## Training data

The curated vision-language component contains roughly 53 million samples. FineVision is its primary general vision-language instruction source and accounts for 24.3 million samples. Other sources in the paper cover visual understanding, localization, planning, spatial reasoning, object affordances, video understanding, and egocentric task comprehension.[1]

Stage 1 combines self-collected Astribot S1 manipulation data with cross-embodiment robot datasets. The named robot sources include AgiBot Genie-1, Galaxea R1 Pro, RealSource RS-02, ARX, Agile X, and YAM platforms. AgiBot World and Galaxea Open-World also appear in the planning portion of the vision-language mixture, while RealSource World supplies dual-arm manipulation data.[5][6][7]

Human video sources include EgoTaskQA, EPIC-KITCHENS, and [Ego4D](/wiki/ego4d).[8][9][10] These datasets contain first-person recordings of daily activities and object interactions. In Stage 1, human video passes only through the visual branch because it does not provide the robot's action representation. Stage 3 adds broader classes of general internet video, egocentric video, self-collected robot video, and cross-embodiment robot video.[1]

The report lists the public raw datasets used in its vision-language mixture, but it does not provide a complete clip-level manifest, per-source mixture weights, total hours for the full video corpus, or the self-collected robot trajectories. The approximately 53 million figure therefore describes the vision-language component, not necessarily the size of the entire mixed corpus.[1]

## Evaluation

All Lumo-2 performance figures were reported by Astribot's research team. The July 2026 release had not been independently replicated.

### Embodied vision-language benchmarks

Astribot evaluated a vision-language-only Lumo-2 checkpoint and the Stage 3 checkpoint with lmms-eval. Visual inputs were limited to 32 images or video frames, video was sampled at 2 FPS, and generation used greedy decoding at temperature 0. Selected comparisons with the Qwen3.5-4B starting model were:[1]

| Benchmark | Lumo-2 VLM | Lumo-2 Stage 3 | Qwen3.5-4B |
| --- | ---: | ---: | ---: |
| VSIBench | 52.34 | 48.56 | 21.73 |
| MindCube | 66.08 | 54.93 | 39.95 |
| MMSI | 36.4 | 35.2 | 30.8 |
| ViewSpatial | 53.08 | 51.15 | 45.47 |
| Where2place | 74 | 70 | 55 |
| RefSpatialBench mean | 61.52 | 57.45 | 44.09 |

The Lumo-2 VLM checkpoint led the compared models on VSIBench, MindCube, Where2place, and RefSpatialBench mean. It was not best on every row in the paper's full table. RynnBrain led ViewSpatial and MMSI, while RoboBrain-2.5 led Ego-Plan2 and several localization or trajectory rows.[1]

### Generalizable pick and place

The pick-and-place suite tested instructions in the form "put A into/to B." Its basic setting used 60 training-seen objects. An unseen-instruction setting required spatial or semantic interpretation, and an unseen-object setting used 105 objects absent from the training data. Lumo-2 was evaluated directly after Stage 3. The compared [pi0.5](/wiki/pi_0_5) model received 10 epochs of Astribot S1 fine-tuning, while Fast-WAM received three.[1][11][12]

The authors reported that Lumo-2 exceeded both baselines on instruction-following rate and success rate across all three settings. The report presents exact results in a figure rather than a text table, so they should not be read as independently verified benchmark scores.[1]

### Latent-world ablation

Astribot compared action-only VLA fine-tuning with VLWA fine-tuning that actively predicted latent world dynamics. The comparison covered two representative temporal-reasoning tasks:[1]

| Task | VLWA fine-tuning | VLA fine-tuning |
| --- | ---: | ---: |
| Collect Eggs from the Conveyor Belt | 100.00% | 92.00% |
| Place Cubes on the Rotating Rack | 81.67% | 74.17% |

These two results support a narrow benefit for the tested temporal tasks. They do not show that the latent-world objective improves every type of robot task.

### Twenty-two real-robot tasks

The main task suite ran on Astribot S1. Each model received matched head-camera observations. The researchers defined 10 scene layouts and evaluated each twice, giving 20 blind trials per model. Single-step tasks used binary success. Multi-step tasks awarded one point for each completed meta-subtask and normalized the total, so partial completion received fractional credit. Several difficult subtasks were evaluated separately from in-domain starting configurations.[1]

| Primary category | Number of tasks | Examples |
| --- | ---: | --- |
| Dynamic scene | 2 | Collect eggs from a conveyor belt; catch a rolling ball |
| Motion reasoning | 4 | Hang cups or place and stack cubes on rotating racks; catch fish |
| Memory | 3 | Erase a whiteboard; pour water; prepare an egg |
| Physical understanding | 4 | Organize a suitcase; scoop millet to a target weight; flip an egg; drive nails |
| Long-horizon execution | 2 | Make coffee; make a cocktail |
| Dexterous manipulation | 7 | Cap pens; unpack a package; pack bags and a suitcase; lift a garbage bag; iron and hang clothes |

Astribot reported that Lumo-2 scored above pi0.5 and Fast-WAM in all six categories. The Fast-WAM baseline received dedicated image augmentation, upscaling, and aspect-ratio adjustment because the researchers found it sensitive to camera-image quality. The results concern these controlled manipulation setups. They do not establish general household autonomy, navigation ability, or safety around untrained people.[1][12]

## Human-to-robot adaptation

The paper evaluates two ways to add human demonstrations during task fine-tuning. In one, an [Apple Vision Pro](/wiki/apple_vision_pro) records first-person RGB video and ARKit 3D hand poses. Astribot transforms the hand poses into the robot base frame, maps wrist poses to the robot's end effectors, and infers gripper commands from thumb-to-index distance. This route supplies vision, language, latent-world, and action supervision.[1]

The second route uses three synchronized egocentric views without action labels. It supervises latent world dynamics but not robot actions. In both cases, the researchers clean and align the human data, mix it with relevant robot demonstrations, fine-tune the model, and test it on robot object settings that appeared only in the human demonstrations. The paper reports higher unseen-object pick-and-place success after adding either human source.[1]

This experiment is a mixed human-robot fine-tuning result. It is not zero-shot transfer from arbitrary video, and it does not show that Lumo-2 can learn an unrestricted task from a single internet clip.

## Inference performance

The paper measured BAR and standard autoregressive decoding with [vLLM](/wiki/vllm), bfloat16 precision, and FlashAttention-2 on one NVIDIA RTX 5090.[13][14] The latency breakdown was:[1]

| Component | Standard AR | BAR |
| --- | ---: | ---: |
| Data processing | 1.33 ms | 1.33 ms |
| Image encoders | 6.49 ms | 6.55 ms |
| Observation forward pass | 18.09 ms | 18.00 ms |
| Action generation | 192.00 ms | 44.96 ms |
| Action detokenization | 5.93 ms | 5.94 ms |
| vLLM scheduler and runtime | 29.82 ms | 16.75 ms |
| Total | 253.66 ms | 93.53 ms |

The total corresponds to a 2.71x speedup in that configuration. It is a single-machine measurement, not a hardware-independent model speed. Astribot's project page says BAR gives this gain without accuracy loss, but the report's BAR section does not provide an autoregressive-versus-BAR control-quality table. The evidence supports the latency comparison, not a general accuracy-equivalence claim.[1][2]

## Representation analysis

Astribot tested whether the learned action representation carried more semantic information. A linear probe classified 18 atomic actions, with 400 examples per class and 7,200 examples total. DINO features alone reached 84.10% top-1 accuracy. Adding Stage 1 action features raised the result to 92.85%, and adding Stage 2 semantic-action features produced 95.00%.[1]

A separate action-reconstruction table contains a discrepancy between its prose and numbers. The report says Stage 2 lowers all rotation and Cartesian errors except gripper channels. Table 6 shows that torso Cartesian MAE increases slightly, from 0.00038 m for the action-only baseline to 0.00043 m for Stage 2. Both gripper errors also increase. Rotation errors and left/right arm endpoint Cartesian errors decrease. The table therefore supports improvement on most listed pose dimensions, not every non-gripper dimension.[1]

## Availability and limitations

The July 2026 project page and report link demonstrations and the technical paper, but neither provides Lumo-2 model weights, training code, an inference repository, a model card, or a model license.[1][2] Some inputs named in the paper are public datasets, while Astribot's self-collected robot data and the complete mixed training corpus are not reproduced in the release.

The report is an arXiv v1 preprint, and all robot experiments were run by the developer on Astribot S1. It does not include a dedicated safety evaluation, collision statistics, energy measurements, deployment-duration tests, or uncontrolled in-home trials. Its long-horizon score can award partial credit for completed meta-subtasks, and some difficult subtasks start from separately prepared in-domain configurations.[1]

The paper names Peijun Tang, Shangjin Xie, Baifu Huang, Binyan Sun, Haotian Yang, Kuncheng Luo, Weiqi Jin, Shilin Fang, and Jianan Wang as authors under the Astribot Team affiliation. It credits Jianan Wang and Peijun Tang with model architecture, and names Jianan Wang as project lead.[1]

## References

1. Peijun Tang, Shangjin Xie, Baifu Huang, Binyan Sun, Haotian Yang, Kuncheng Luo, Weiqi Jin, Shilin Fang, and Jianan Wang, Astribot Team, "Towards Predictive, Aligned, and Scalable Robot Learning," arXiv:2607.11270v1, July 13, 2026. https://arxiv.org/abs/2607.11270
2. Astribot, "Lumo-2: Latent World-Action Model for Predictive, Aligned, and Scalable Robot Learning," official project page, July 2026. https://www.astribot.com/research/Lumo2
3. Peijun Tang et al., Astribot Team, "Mind to Hand: Purposeful Robotic Control via Embodied Reasoning," arXiv:2512.08580, December 2025. https://arxiv.org/abs/2512.08580
4. Maxime Oquab et al., "DINOv2: Learning Robust Visual Features without Supervision," arXiv:2304.07193, April 2023. https://arxiv.org/abs/2304.07193
5. Q. Bu et al., "AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems," arXiv:2503.06669, March 2025. https://arxiv.org/abs/2503.06669
6. Tao Jiang et al., "Galaxea Open-World Dataset and G0 Dual-System VLA Model," arXiv:2509.00576, August 2025. https://arxiv.org/abs/2509.00576
7. RealSource, "RealSource World: A Large-Scale Real-World Dual-Arm Manipulation Dataset," Hugging Face dataset card, 2025. https://huggingface.co/datasets/RealSourceData/RealSource-World
8. Baoxiong Jia, Ting Lei, Song-Chun Zhu, and Siyuan Huang, "EgoTaskQA: Understanding Human Tasks in Egocentric Videos," arXiv:2210.03929, October 2022. https://arxiv.org/abs/2210.03929
9. Dima Damen et al., "The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines," arXiv:2005.00343, April 2020. https://arxiv.org/abs/2005.00343
10. Kristen Grauman et al., "Ego4D: Around the World in 3,000 Hours of Egocentric Video," arXiv:2110.07058, October 2021. https://arxiv.org/abs/2110.07058
11. Physical Intelligence et al., "pi0.5: a Vision-Language-Action Model with Open-World Generalization," arXiv:2504.16054, April 2025. https://arxiv.org/abs/2504.16054
12. Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao, "Fast-WAM: Do World Action Models Need Test-time Future Imagination?", arXiv:2603.16666, March 2026. https://arxiv.org/abs/2603.16666
13. Woosuk Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention," arXiv:2309.06180, September 2023. https://arxiv.org/abs/2309.06180
14. Tri Dao, "FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning," arXiv:2307.08691, July 2023. https://arxiv.org/abs/2307.08691
15. Qwen Team, "Qwen3.5-4B," first-party model card and weights repository, 2026. https://huggingface.co/Qwen/Qwen3.5-4B
16. Guang Gao et al., Astribot Team, "Towards Human-level Intelligence via Human-like Whole-Body Manipulation," arXiv:2507.17141, July 2025. https://arxiv.org/abs/2507.17141
17. RobotStart, "Astribot Lumo-2" introductory-video coverage, July 16, 2026. https://robotstart.info/article/2026/07/16/382157.html
18. Robot Lecture Hall, "Astribot Releases Lumo-2," technical coverage, July 15, 2026. https://www.leaderobot.com/news/8443