# RoboChallenge

> Source: https://aiwiki.ai/wiki/robochallenge
> Updated: 2026-11-01
> Fact-checked: 2026-11-01
> Categories: AI Benchmarks, Embodied AI, Model Evaluation, Robotics
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "RoboChallenge." aiwiki.ai, 1 Nov 2026. https://aiwiki.ai/wiki/robochallenge
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**RoboChallenge** is an online benchmarking platform that evaluates robot control policies, mainly [vision-language-action models](https://aiwiki.ai/wiki/vision_language_action_model) (VLAs), on physical robots instead of in simulation. It was started by the Chinese embodied-AI company Dexmal together with [Hugging Face](https://aiwiki.ai/wiki/hugging_face) and went live in mid-October 2025 with its first benchmark, **Table30**, a set of 30 tabletop manipulation tasks run on UR5, Franka, ARX-5 and ALOHA arms.[1][2][3] Participants keep their models on their own computers and drive the platform's robots remotely through an API, while staff at the testing site reset the scenes and grade each attempt. Every model is scored on two metrics, an end-to-end success rate and a staged progress score.[1][3] The platform's organizers describe it as the first large-scale real-robot benchmark for embodied intelligence.[3] Table30 was retired on 27 May 2026 and replaced by **Table30 V2**, which requires a single multi-task model and adds a dual-arm DOS-W1 robot.[2][8][13]

## Background

Many earlier VLA benchmarks run in simulation; the RoboChallenge technical report cites LIBERO, CALVIN and RoboTwin among them. The technical report argues that real-robot testing is still necessary because "the real-world always contains factors that the digital twin cannot reproduce". It also notes that the existing online real-robot evaluators it cites, RoboArena and AutoEval, host only a few tasks on one or two machines.[1] In an October 2025 conversation published by QbitAI, Dexmal co-founder and CEO Tang Wenbin said that many methods work perfectly in simulation but fail in the real world, and that the field lacked a unified, open and reproducible real-robot benchmark for comparing models. In the same discussion Hugging Face co-founder Thomas Wolf drew on Hugging Face's experience running the Open LLM Leaderboard, arguing that an independent platform running standardized, reproducible tests helps the community filter out noise such as marketing videos and cherry-picked demos.[11]

Dexmal (Chinese: 原力灵机) is a Chinese [embodied AI](https://aiwiki.ai/wiki/embodied_ai) startup. According to QbitAI it was registered at the end of 2024 and formally established on 5 March 2025, and about 20 days later it announced a 200 million yuan angel round from investors including Legend Capital, Jiukun Venture and Qiming Venture Partners. Tang Wenbin is a co-founder of [Megvii](https://aiwiki.ai/wiki/megvii), where he was CTO, and core team members Fan Haoqiang, Zhou Erjin and Wang Tiancai also came from Megvii.[11] RoboChallenge's 2025 annual report credits the Dexmal team with providing the real-robot cluster and the long-term operating staff for the platform.[3]

## History

| Date | Event |
|---|---|
| 14 October 2025 | RoboChallengeInference repository created on GitHub[24] |
| 15 October 2025 | Launch date given in the annual report; press coverage the same day[3][10] |
| 17 October 2025 | "We are online!" post on robochallenge.ai with initial Table30 baselines[2] |
| 20 October 2025 | Technical report posted to arXiv (2510.17950)[1] |
| 20 November 2025 | RoboChallenge Committee launched with ten partners[2][3] |
| 28 November 2025 | Baseline inference code and a fine-tuned π0.5 model released for the stack_bowls task[2] |
| 25 December 2025 | Community page opened, inviting organizations to join the working groups[2] |
| 13 January 2026 | Xinhua reports Spirit AI's Spirit v1.5 ranked first on Table30[16] |
| 14 January 2026 | Per-rollout server-side and robot-side logs made available for debugging[2] |
| 30 January 2026 | Annual report (2025 Q4 to 2026 Q1) published[2][3] |
| 24 March 2026 | Table30 V2 announced, with a preview used for a CVPR 2026 workshop competition[13] |
| April 2026 | Second group of eight partner companies joins[14] |
| 18 May 2026 | CVPR 2026 workshop competition results posted[21] |
| 21 May 2026 | QbitAI reports RobotEra's Era0 at the top of Table30[15] |
| 27 May 2026 | Table30 retired; Table30 V2 takes over[2] |
| 28 May 2026 | ICRA 2026 competition results posted[22] |
| 19 August 2026 | Simulation collaboration with Lightwheel and NVIDIA announced[2][20] |

The launch was covered by Chinese technology outlets including Zhidx (republished by ifeng.com) and AIbase, both of which described RoboChallenge as jointly initiated by Dexmal and Hugging Face.[10][12] The platform says that during Table30's run it carried out more than 80,000 real-world robot task executions and put more than 20 models on the leaderboard.[2]

## The RoboChallenge system

### Remote robot interface

The technical report compares three ways of exposing robots to outside users: submitting model weights for the organizer to run, submitting a Docker image containing the whole inference system, and the model API call approach used by RoboArena, in which the evaluator calls a URL provided by the user. RoboChallenge uses none of them. In what the authors call the "remote robot" paradigm, the model always runs on the user's side, and the user is responsible for all format conversion and action post-processing.[1]

The user sends capture requests to receive precisely timestamped observations (RGB, depth and proprioception) and posts actions, each with a duration, into an action queue on the robot. The robot executes the queue in first-in, first-out order and reports the current queue length through the API. Actions already queued cannot be revoked. Because users call RoboChallenge's API rather than exposing their own, they do not need a public IP address. The authors gave three reasons for the design: getting a submitted model to run on someone else's software and hardware stack is very hard to debug; it avoids forcing a "stop-and-inference" control loop on methods such as real-time action chunking that need exact timestamps; and many users sit behind network address translation. Separate scheduling APIs tell users when their job will run, so they can load weights and warm up a GPU minutes before the evaluation starts.[1]

### Robots

At the first benchmark's release the platform offered four robot types, picked for durability, popularity in research, safety and control performance (cyclic position control up to 100 Hz and millimeter-level repeatability):[1]

| Robot | Configuration | Notes from the technical report |
|---|---|---|
| [Universal Robots](https://aiwiki.ai/wiki/universal_robots) UR5 | Single 6-DOF arm with a Robotiq gripper | Controlled through RTDE at up to 125 Hz; described as extremely durable |
| Franka Panda | 7-DOF arm, gripper replaced by a Robotiq unit | Joint and end-point control modes through libfranka |
| Cobot Magic ALOHA | Two 6-DOF arms on a moving platform that mimics the [ALOHA](https://aiwiki.ai/wiki/aloha_robot) system | Cheaper, higher failure rate; vendor CAN drivers |
| ARX-5 | Single 6-DOF arm mounted on a table | Cheaper, higher failure rate; vendor CAN drivers |

Each robot uses several Intel RealSense RGB-D cameras: a main camera looking down on the work area, a wrist camera, and an extra side camera on single-arm setups. Torque and force sensing was left out at launch. When hardware fails during a run, the evaluation is revoked and resumed after maintenance.[1] The report describes an initial fleet of 10 machines; by the annual report's January 2026 cut-off the testing cluster had 20 real robots.[1][3]

### Evaluation protocol

A large part of the technical report deals with run-to-run variation, which the authors say can swing a measured success rate from 0% to 100% with the same props, task and model. In one experiment they had three kinds of human tester run the same two tasks: "experienced" testers who had collected the training data, "ignorant" testers seeing the task for the first time, and "adaptive" testers, who were the model authors. The adaptive testers got better results by exploiting what the report calls the "sweet-spot effect", placing objects where the model was more likely to succeed, while the ignorant testers' results were unstable.[1]

RoboChallenge's answer is **visual task reproduction**. Some episodes from each task's demonstration data are held out as reference episodes. Before each rollout, the initial frame of a reference episode is superimposed on the tester's live camera view, and the tester moves the objects until the live image matches it. The authors report that this "controlled tester" method was empirically more stable than using experienced testers, and that it does not require testers to know how the training data were collected.[1] Remaining factors such as day-to-day lighting changes and slow camera drift are treated as intended perturbation; a small experiment in the report found that VLA outputs changed little when input images were manually augmented or partly occluded.[1]

The report separates **stability** (how reproducible one model's result is) from **fairness** (how stable the relative order of models is). The protocol RoboChallenge runs, the "benchmark protocol", targets stability. The authors also proposed a "comparative protocol" in which the tester prepares a scene, a randomly selected model from a group is run, and the tester does not know which model is running, but at publication the platform served only the benchmark protocol.[1]

The report lists its own known limitations. Because inference happens on the user's side, the platform cannot check that the model a user runs matches the model name they claim; a user could in principle use individually tuned models where a generalist is expected, or even cheat with a human in the loop. The authors ask users to release models and code so results can be reproduced. They also note the risk of overfitting to the fixed reference test cases, which they had not observed in practice.[1] The annual report repeats the point that the platform "objectively cannot verify the authenticity of users' proprietary model architectures" and adds that slight lighting changes, or even the color of an operator's clothing, can shift a model's score between two submissions. It says a secondary scoring review and an appeal channel were introduced to correct grading errors.[3]

### Submission workflow

Participants download the demonstration data from Hugging Face, where it is published as video files with separate JSON robot states plus a script to convert it to [LeRobot](https://aiwiki.ai/wiki/lerobot) format. They fine-tune their model, adapt RoboChallenge's skeleton client code (an observe, infer and wait cycle with a mock test) and submit an evaluation request with their key, the chosen tasks and a model name. Submissions are queued manually, and the report warns that the wait can be hours to days because props and testers must be prepared. Results and videos are published on the website, machine logs can be viewed in RRD format with the open-source rerun.io viewer, and by default all participants can see each other's results.[1] The example client code is published in the RoboChallengeInference repository on GitHub, which had 158 stars as of 1 November 2026.[24]

## Table30

### Tasks

Table30 consists of 30 tasks performed on or around a table by static arms. The technical report says the tasks were chosen to cover a range of difficulty, a spread of algorithmic challenges and everyday scenes from homes, restaurants and workplaces, while staying simple enough that a human could do them without training. It calls the benchmark a "necessity test" for general robotics.[1] The annual report groups the tasks into 9 categories.[3]

| Task | Robot | Description (from the technical report) |
|---|---|---|
| arrange flowers | ARX5 | Pick up three flowers and insert them into the vase |
| arrange fruits in basket | UR5e | Put four fruits into the basket |
| arrange paper cups | ARX5 | Stack five paper cups and put them on a shelf |
| clean dining table | ALOHA | Put trash and dishes into a trash bin and basket |
| fold dishcloth | ARX5 | Fold the dishcloth twice and put it to the side |
| hang toothbrush cup | UR5 | Hang a cup on the cup holder |
| make vegetarian sandwich | ALOHA | Make a vegetable sandwich |
| open the drawer | ARX5 | Open the drawer |
| place shoes on rack | ARX5 | Place a pair of shoes on the shoe rack |
| plug in network cable | ALOHA | Insert two RJ45 connectors into the socket |
| pour fries into plate | ALOHA | Open the box and pour the fries onto the plate |
| press three buttons | Franka | Press the pink, blue and green buttons in sequence |
| put cup on coaster | ARX5 | Place the cup on the coaster |
| put opener in drawer | ALOHA | Place the can opener in the right-hand drawer |
| put pen into pencil case | ALOHA | Put the pen into the pencil case |
| scan QR code | ALOHA | Scan the QR code on a medicine box with a scanner |
| search green boxes | ARX5 | Pick all green boxes out of a pile into a yellow box |
| set the plates | UR5 | Place three plates onto the plate rack one by one |
| shred scrap paper | UR5 | Stuff paper into the shredder |
| sort books | UR5 | Place three books in their positions on a shelf |
| sort electronic products | ARX5 | Put four electronic products into four baskets |
| stack bowls | ALOHA | Stack three bowls together |
| stack color blocks | UR5 | Stack the yellow block on the orange block |
| stick transparent tape to box | ALOHA | Tear off clear tape and stick it onto the box |
| sweep the rubbish | ALOHA | Sweep trash into the dustpan with a broom |
| move objects into box | Franka | Put all the clutter on the desk into a white basket |
| turn on faucet | ALOHA | Grasp the faucet switch and turn it on |
| turn on light switch | ARX5 | Turn on the light switch |
| water potted plant | ARX5 | Water the potted plant with the kettle |
| wipe the table | ARX5 | Wipe stains off the table with a tissue and discard it |

Each task carries tags describing its difficulty for a VLA: temporal (identical images appear at different stages), softbody, precise3d, bimanual, multiview, repeated (a skill repeated at least three times), classification, manipulation (hinges or racks) and simple-pick.[1][7] For each task RoboChallenge provides demonstration data, up to 1,000 episodes per task.[1][3] The annual report says the Table30 dataset on Hugging Face had been downloaded about 17,000 times by its January 2026 cut-off.[3]

### Scoring

Each task is divided into stages worth a set number of progress points, totalling 10 per rollout. Stages can be marked non-critical, meaning the task still counts as a success if that stage is missed. The report's example for opening a drawer awards 2 points for reaching the drawer region, 3 for rotating the gripper toward the handle, 4 for pulling the drawer open (all critical) and 1 for returning the arm (not critical). Each retry of a stage costs 0.5 points, and a rollout is stopped if a stage's score goes negative or more than four successive retries fail. Every model gets 10 rollouts per task, so a task's progress score is out of 100.[1]

A model's leaderboard **success rate** is the mean of its 30 per-task success rates, and its **score** is the mean of its 30 per-task progress scores. Only models that complete all 30 tasks qualify for the overall ranking, which is sorted by success rate because the organizers consider it the best measure of whether a model actually solves real tasks.[3] Because retries are penalized and partial progress is rewarded, a task can succeed with a low progress score or fail with a high one.[1]

The platform distinguishes **single-task (specialist)** entries, where a separate model may be fine-tuned for each task, from **multi-task (generalist)** entries that handle 2 to 30 tasks with one model.[3] In the technical report's own baselines the task-specific setting used all of a task's demonstration data and typically took one day on an 8-GPU machine, while the generalist setting mixed about 50 episodes from each task, and only from tasks on the same robot type, making the result a "machine generalist".[1] Entries from the same user that share a display name, for example task-specific fine-tunes of one base model, are grouped into a single leaderboard row.[1]

### Initial results

The technical report surveyed four open-source VLA algorithms: [π0](https://aiwiki.ai/wiki/pi0) and [π0.5](https://aiwiki.ai/wiki/pi_0_5) from [Physical Intelligence](https://aiwiki.ai/wiki/physical_intelligence), Microsoft's CogACT, and OpenVLA/OFT, derived from [OpenVLA](https://aiwiki.ai/wiki/openvla). The entries marked "rc_baseline" were fine-tuned by the report's authors and the others by college volunteers.[1] Its per-task results table gives these averages:[1]

| Entry | Setting | Average success rate (%) | Average progress score |
|---|---|---|---|
| π0.5 | Task-specific | 43.7 | 62.2 |
| π0 | Task-specific | 28.3 | 47.6 |
| π0.5/multi | Generalist | 17.7 | 31.3 |
| CogACT | Task-specific | 11.7 | 21.8 |
| π0/multi | Generalist | 9.3 | 20.6 |

The authors concluded that fine-tuned π0.5 was "remarkably stronger in almost all respects", while noting that π0.5 trained as a generalist on only about 50 episodes per task sometimes beat the task-specific model. Averaged over all models, tasks tagged temporal (5% success) and softbody (8%) were hardest and simple-pick tasks easiest (42%); the authors attributed the temporal failures to all tested models being single-frame models, and the difficulty of precise3d tasks partly to the models' 224x224 input resolution.[1]

### Leaderboard

The Table30 leaderboard changed hands several times. Xinhua reported on 13 January 2026 that, according to the leaderboard, [Spirit AI](https://aiwiki.ai/wiki/spirit_ai)'s Spirit v1.5 ranked first with a score of 66.09 and a 50.33% success rate, ahead of π0.5, and was the only model above 50% success at the time.[16] [GigaAI](https://aiwiki.ai/wiki/gigaai)'s GigaBrain-0.5M* paper states that an intermediate model, GigaBrain-0.1, ranked first as of 9 February 2026 with a 51.67% average success rate, while a 12 February report from China's Optics Valley described GigaBrain-0.1 as second.[17][18] Dexmal's own DM0 paper, posted on 16 February 2026, claims state-of-the-art results on Table30 in both the specialist and generalist settings.[19] On 21 May 2026 QbitAI reported that [RobotEra](https://aiwiki.ai/wiki/robotera)'s Era0 had taken first place with a 64.33% success rate and a 76.34 score, and that Era0 had the best result on 17 of the 30 tasks.[15]

The final Table30 leaderboard, as published on 1 November 2026, lists 22 entries:[5][23]

| Rank | Model | Submitter (as listed) | Multi-task | Score | Success rate |
|---|---|---|---|---|---|
| 1 | Era0 | Robotera | No | 76.34 | 64.33% |
| 2 | DM0 | Dexmal | No | 72.25 | 62.00% |
| 3 | GigaBrain-0.1 | lyf | No | 68.34 | 51.67% |
| 4 | Spirit-v1.5 | Spirit AI | No | 67.19 | 51.00% |
| 5 | Lira_generalist | Atlas | Yes | 59.83 | 45.00% |
| 6 | pi0.5 | rc_baseline | No | 61.84 | 42.67% |
| 7 | Atlas_generalist | Atlas | Yes | 55.10 | 39.67% |
| 8 | DM0_generalist | Dexmal | Yes | 49.08 | 37.33% |
| 9 | wall-oss-v0.1 | Pushi Zhang | No | 55.30 | 35.33% |
| 10 | pi0 | rc_baseline | No | 46.41 | 28.33% |
| 11 | VLO | Sicheng Xie | Yes | 42.34 | 25.67% |
| 12 | X-VLA | Chongyang Xu | No | 34.75 | 21.33% |
| 13 | pi05_generalist | wyf | Yes | 31.27 | 17.67% |
| 14 | RDT-1B | zsz | No | 28.84 | 15.00% |
| 15 | GR00T | Sicheng Xie | No | 31.64 | 14.33% |
| 16 | GR00T-MULTI | Sicheng Xie | Yes | 28.39 | 13.33% |
| 17 | ZR-0_generalist | Atlas | No | 29.20 | 12.67% |
| 18 | cogact | hsk | No | 21.83 | 11.67% |
| 19 | VLA_test_0204 | Xiangyue Wu | Yes | 17.33 | 9.00% |
| 20 | pi0_generalist | wyf | Yes | 20.22 | 9.00% |
| 21 | VLA_test_0119 | Xiangyue Wu | Yes | 21.95 | 5.00% |
| 22 | openvla-oft | gkf | No | 8.66 | 5.00% |

The board is ordered by success rate, so ranks differ when sorted by progress score; π0.5 (rc_baseline), for example, has a higher score than the multi-task Lira_generalist but a lower success rate. "Multi-task" reflects the flag each entry declared on the board.[5][23] Spirit-v1.5's current figures (67.19 and 51.00%) are slightly higher than those reported in January.[16][23] By January 2026, according to the annual report, the Spirit AI and wall-oss ([X Square Robot](https://aiwiki.ai/wiki/x_square_robot)) teams had completed the full Table30 evaluation, and the multi-task board held only two small-data reference models, pi05_generalist and pi0_generalist (about 50 samples per task), which the annual report describes as RoboChallenge's own baselines although the leaderboard lists them under the user wyf. On those baselines, π0.5's success rate fell from 42.67% as a single-task model to 17.67% as a multi-task model, and π0's from 28.33% to 9.00%.[3]

The challenges page reports 3,985 runs and 25,627 episodes on Table30.[6] The annual report's analysis, based on results as of January 2026, called stack_bowls and move_objects_into_box "Hello World" tasks, listed make_vegetarian_sandwich and water_potted_plant among tasks on which every listed model scored 0%, and said success rates on fine-manipulation tasks were below 15%.[3] QbitAI later reported that Era0 was the only top-eight model with a non-zero success rate on make_vegetarian_sandwich (20%) and wipe_the_table (60%).[15]

## Table30 V2

RoboChallenge announced Table30 V2 on 24 March 2026 under the slogan "large-scale real-device native generalization evaluation for next-generation models", and its benchmark page shows a last update of 28 May 2026, the day after Table30's retirement.[13][8][2] According to the announcement, which QbitAI republished in its partner-content column in the organizers' own voice:[13]

- **Tasks:** 12 tasks kept from Table30 plus 18 new dual-arm dexterous tasks, including deformable objects such as rope and cloth and tool use.
- **Hardware:** a new mobile dual-arm platform, the DOS-W1, alongside the ALOHA.
- **Evaluation:** full support for multi-task models, with entrants asked to submit a single model rather than a separate model per task; object-level and environment-level zero-shot tests; and in-domain versus out-of-domain testing.
- **System:** about three times the previous system throughput through more robots and better scheduling, a looser "rough alignment" of initial scenes instead of pixel-level matching, and a completion-time metric on the leaderboard (as of 1 November 2026 the published leaderboard shows only score and success rate).

The Table30 V2 dataset card lists 30 tasks across four embodiments: the single-arm ARX5 (wrist, global and side cameras) and UR5 (wrist and global cameras), and the dual-arm ALOHA and DOS-W1 (two wrist cameras and a global camera).[9] On the submission page, the tasks are split by robot: ten on the DOS-W1, ten on the ALOHA, seven on the ARX5 and three on the UR5. Only tasks submitted together as a unified test for one embodiment count toward results.[25] New tasks include folding clothes, tying a knot, untying shoelaces, spreading jam, using a lint roller and stamping a signature area.[8][9]

As of 1 November 2026 the challenges page reports 1,105 runs, 32,980 episodes and 58 models on Table30 V2.[6] The leaderboard's published figures are averages over all 30 tasks, with tasks an entry has not run counting as zero; LoopWAM's leading 43.33%, for example, comes from results on 25 tasks, while VLA-DM0.5 has results on all 30.[23] The top of the board on that date:[5]

| Rank | Model | Submitter (as listed) | Score | Success rate |
|---|---|---|---|---|
| 1 | LoopWAM | LoopWAM | 56.80 | 43.33% |
| 2 | VLA-DM0.5 | KDDI Research | 54.42 | 40.67% |
| 3 | ambase | AVerse001 | 43.67 | 31.67% |
| 4 | my16 | Tymtbo | 41.27 | 30.67% |
| 5 | JV0 | JIIOV | 27.58 | 16.00% |
| 6 | mc | mcBrains | 23.63 | 15.33% |
| 7 | MagicBot | MagicBot | 20.82 | 14.67% |
| 8 | Baseline Model | RoboChallenge | 31.48 | 14.33% |
| 9 | holobrain | HRL | 27.93 | 14.33% |
| 10 | test_01 | VLA | 22.73 | 12.33% |

### Simulation counterpart

On 19 August 2026 RoboChallenge announced a collaboration with Lightwheel and [NVIDIA](https://aiwiki.ai/wiki/nvidia) to build a simulation version of Table30 V2 in [NVIDIA Isaac Lab](https://aiwiki.ai/wiki/isaac_lab), with task definitions, metrics, success criteria and scoring aligned with the real-robot benchmark to support [sim-to-real](https://aiwiki.ai/wiki/sim_to_real_transfer) research. Lightwheel and Dexmal are to build the simulation assets, and a new Simulation Working Group is jointly led by Lightwheel and RoboChallenge. The organizers say future competitions will release tasks in Isaac Lab first, with real-robot validation for top teams, and that they are exploring an industrial benchmark series called the "Real Series".[2][20]

## Competitions

RoboChallenge has hosted two competitions on its robots:

| Competition | Format | Dates | Winner (score, success rate) |
|---|---|---|---|
| CVPR 2026 Workshop, GigaBrain Challenge, RoboChallenge track | Single multi-task model on a "Table30 CVPR version"; fully remote; prize pool up to $6,000 for the top category | Competition 15 April to 15 May 2026; results 18 May 2026 | my16 (Tymtbo), 55.02, 40.89%[21] |
| ICRA 2026 RoboChallenge competition | Supermarket scenario covering instruction following, navigation, picking and restocking (Dexmal whole-body-control track) | Competition 8 to 25 May 2026; results 28 May 2026 | "my grasper" (周熊队), 99.20, 94.00%[22] |

The CVPR track was part of the GigaBrain Challenge 2026 and served as the first public outing of the Table30 V2 preview.[13][21] Prizes in both competitions are paid in renminbi; the ICRA prizes include cash and vouchers issued by [AgiBot](https://aiwiki.ai/wiki/agibot).[21][22]

## Governance and partners

The RoboChallenge Committee was established on 20 November 2025. Initiated by Dexmal and Hugging Face, its first partners were the [Beijing Academy of Artificial Intelligence](https://aiwiki.ai/wiki/baai), AgiBot, [Qwen](https://aiwiki.ai/wiki/qwen), Galaxea ([Galaxea Dynamics](https://aiwiki.ai/wiki/galaxea_dynamics)), X Square Robot, [Tsinghua University](https://aiwiki.ai/wiki/tsinghua_university), Xi'an Jiaotong University and GOSIM.[3] Under its governance model, partners form a Steering Committee that sets strategy and approves major initiatives, and four working groups handle benchmark and task design, lab maintenance (accepting donated robots and running tests), platform development, and community engagement. Decisions follow a consensus model, and results, datasets, code and evaluation reports are to be published on robochallenge.ai, GitHub and Hugging Face under open licenses.[4]

By 23 January 2026 the committee had received membership applications from organizations including China Mobile (Hangzhou) Information Technology, Lightwheel, Spirit AI, Robbyant and GigaAI, and Galaxea had donated an R1 Lite robot to the lab.[3] In April 2026 a second group of eight companies joined, including RobotEra, GigaAI, [Horizon Robotics](https://aiwiki.ai/wiki/horizon_robotics), Lightwheel, Shengshu Technology, China Mobile (Hangzhou) and Changhong, with announced plans such as a Lightwheel simulation suite for Table30, a Horizon Robotics instruction-following benchmark called AnyMove, a restaurant benchmark from China Mobile (Hangzhou) and a household benchmark from a Changhong-led training base in Sichuan.[14]

The annual report gives these platform statistics for its October 2025 to January 2026 period: 209 testing-token applications, of which 82 developers went on to submit evaluations; 41,969 real-robot rollouts in total; and peaks of 181 evaluation runs and 834 rollouts in a single day. Active users were 58.3% from China, 22.0% from the United States, 10.1% from Singapore, 7.6% from Japan and 2.0% from the UAE. The report says the platform was designed for 800 rollouts a day but saw daily demand of 1,000 to 1,500 in December 2025, and that scheduling changes and longer service hours raised measured capacity by 50%.[3]

## Reception and limitations

Chinese technology media have covered RoboChallenge results as a reference point for embodied-AI companies. Spirit AI, GigaAI and Dexmal cited their Table30 results in announcements or papers, and QbitAI covered RobotEra's lead.[15][16][17][19] Two of the articles describing the platform in the strongest terms appeared in QbitAI's partner-content column, and the Table30 V2 announcement is written in the organizers' voice.[13][14]

The platform's own documents name its main weaknesses. Remote, user-side inference means RoboChallenge cannot confirm what model a participant actually ran, including whether a "generalist" entry is really one model. Results can shift between runs because of uncontrolled physical conditions, and grading by human testers needs secondary review.[1][3] The specialist setting also lets entrants fine-tune a separate model for each task with up to 1,000 demonstrations each, so specialist and multi-task results are not directly comparable; the annual report's single-task versus multi-task baseline gap illustrates the difference.[1][3] Table30 V2's move to single multi-task models and zero-shot tests was presented as a response to this kind of per-task tuning.[13] The annual report also lists community requests for zero-shot benchmarks, simulation counterparts of the real tasks, more robot types and a transparent submission queue.[3]

## Related benchmarks

- [RoboArena](https://aiwiki.ai/wiki/roboarena): crowd-sourced real-world evaluation in which a distributed network of evaluators runs double-blind pairwise comparisons of policies on the DROID platform; cited in the RoboChallenge report as an earlier online real-robot evaluator.[1][26]
- [RoboDojo](https://aiwiki.ai/wiki/robodojo): a unified sim-and-real benchmark for generalist robot manipulation policies.[27]
- [LIBERO](https://aiwiki.ai/wiki/libero) and [RoboCasa](https://aiwiki.ai/wiki/robocasa): simulation benchmarks for robot manipulation.

## References

1. Yakefu, A., Xie, B., Xu, C., et al. "RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies." arXiv:2510.17950, 20 October 2025. https://arxiv.org/abs/2510.17950
2. RoboChallenge. "News." robochallenge.ai. Accessed 1 November 2026. https://robochallenge.ai/news
3. RoboChallenge Committee. "RoboChallenge Annual Report, 2025 Q4 to 2026 Q1." January 2026. https://robochallenge.ai/2025%20RoboChallenge%20Annual%20Report.pdf
4. RoboChallenge. "RoboChallenge Committee Governance Model." https://robochallenge.ai/RoboChallenge_Committee_Governance_Model.pdf
5. RoboChallenge. "Leaderboard." Accessed 1 November 2026. https://robochallenge.ai/leaderboard
6. RoboChallenge. "Challenges." Accessed 1 November 2026. https://robochallenge.ai/challenges
7. RoboChallenge. "Table 30" benchmark detail. https://robochallenge.ai/benchmark_detail
8. RoboChallenge. "Table 30 V2" benchmark detail. https://robochallenge.ai/benchmark_detail/table30_v2
9. RoboChallenge. "RoboChallenge Table30 v2 Dataset." Hugging Face. https://huggingface.co/datasets/RoboChallenge/Table30v2
10. Zhidx (智东西). "全球首个！RoboChalleng登场，开启大规模机器人实体评测." ifeng.com, 15 October 2025. https://tech.ifeng.com/c/8nUclS0NPDX
11. QbitAI (量子位). "开源对机器人的价值，远超大模型时代的想象丨唐文斌深度对谈抱抱脸创始人." Republished by BAAI Hub, 21 October 2025. https://hub.baai.ac.cn/view/49656
12. AIbase. "RoboChallenge, the World's First Real-Physical-Environment Multi-Task Benchmark, is Released." 16 October 2025. https://news.aibase.com/news/22008
13. QbitAI (量子位, partner-content column). "你的模型真的会'举一反三'吗？RoboChallenge Table30 V2 正式发布，泛化时代开幕." 24 March 2026. https://www.qbitai.com/2026/03/391744.html
14. QbitAI (量子位, partner-content column). "18家具身顶尖势力集结，RoboChallenge 打造全球最大具身模型竞技场." 17 April 2026. https://www.qbitai.com/2026/04/402186.html
15. Tian Yanlin. "顺丰邮政仓库干活的机器人，顺手拿了个具身高考第一." QbitAI, 21 May 2026. https://www.qbitai.com/2026/05/420918.html
16. Xinhua / APP. "Chinese AI startup tops global embodied intelligence benchmark." 13 January 2026. https://www.app.com.pk/global/chinese-ai-startup-tops-global-embodied-intelligence-benchmark/
17. GigaAI. "GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learning." arXiv:2602.12099, February 2026. https://arxiv.org/abs/2602.12099
18. Optics Valley of China. "OVC's 'Robot Brain' ranks second globally on RoboChallenge platform." China Services Info, 12 February 2026. https://www.chinaservicesinfo.com/s/202602/12/WS69a16430498e23165e06db44/ovcs-robot-brain-ranks-second-globally-on-robochallenge-platform.html
19. Yu, E., Lv, H., Sun, J., et al. "DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI." arXiv:2602.14974, 16 February 2026. https://arxiv.org/abs/2602.14974
20. RoboChallenge. "RoboChallenge, Lightwheel, and NVIDIA Collaborate to Bridge Simulation and Reality for Embodied AI Evaluation." https://robochallenge.ai/simulation/table30_v2
21. RoboChallenge. "CVPR 2026 Workshop Competition." https://robochallenge.ai/competition/cvpr
22. RoboChallenge. "ICRA Competition 2026." https://robochallenge.ai/competition/icra
23. RoboChallenge. Table30 and Table30 V2 leaderboard data (JSON). Accessed 1 November 2026. https://robochallenge.ai/api/v1/leaderboard/leaderboard_all.json and https://robochallenge.ai/api/v2/leaderboard/leaderboard_table30_v2.json
24. RoboChallenge. "RoboChallengeInference." GitHub. https://github.com/RoboChallenge/RoboChallengeInference
25. RoboChallenge. "Eval Your Policy." https://robochallenge.ai/evalyourpolicy
26. Atreya, P., Pertsch, K., Lee, T., et al. "RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies." arXiv:2506.18123, 2025. https://arxiv.org/abs/2506.18123
27. "RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies." arXiv:2607.04434, 2026. https://arxiv.org/abs/2607.04434

