Qwen-RobotManip：对齐解锁机器人操作基础模型的规模化能力

Qwen-Omni × Qwen-RobotManip — Qwen-Omni observes the scene, randomly proposes manipulation tasks via speech, and judges execution in real time. Each video shows Qwen-RobotManip completing tasks on the fly with no pre-defined task list, demonstrating open-ended instruction following and generalization.

Qwen-RobotManip is validated across various real-robot platforms and tasks, demonstrating strong generalization to novel scenes, unseen language instructions, and cross-embodiment transfer.

View Real-World Evaluation Gallery

Foundation models in language and multimodality have achieved remarkable generalization because heterogeneous data sources can be aligned under a unified formulation, and abundant low-cost internet data allows diverse training signals to reinforce one another at scale. But can this scaling recipe be applied to robotic manipulation?

This is challenging. Unlike text or images, manipulation data is heterogeneous by nature, expensive to collect, and narrow in diversity. Aligning representations across different robot embodiments, sensors, and task domains while simultaneously scaling the data has remained an open problem.

Qwen-RobotManip is a generalizable Vision-Language-Action (VLA) foundation model built upon Qwen-VL. It introduces a unified alignment framework across the representation, motion, and behavioral dimensions of manipulation, making large-scale multi-source training coherent rather than conflicting. Using only open-source robotic manipulation datasets and human demonstration videos without any proprietary data collection, Qwen-RobotManip constructs a ~38,100 hours pretraining corpus and already exhibits emergent generalization capabilities.

Without unified cross-embodiment alignment, scaling data produces conflicts; without data diversity, alignment alone cannot generalize. Alignment and scale are tightly coupled prerequisites for robotic foundation models.

Key Highlights#

Alignment

Representation · Motion · Behavior

Three-Dimensional Alignment

Open-Source Data Only

38K Hours of Manipulation Data

Across 15 Embodiments

Dominant

OOD Generalization

Across All Benchmarks

#1

RoboChallenge Table30 v1 Generalist Track

Sweeping Top 2, 20% Ahead of 3rd Place

Unified Cross-Embodiment Alignment Framework — a unified 80-dimensional state-action representation accommodates diverse embodiments, camera-frame end-effector delta poses make visually similar motions numerically proximate, and in-context policy adaptation reads execution history as an implicit embodiment identifier — together enabling consistent signal extraction across embodiments Human-to-Robot Synthesis at Scale — a pipeline converting 1,933h of egocentric human video into 24,808h of robot demonstrations across 15 embodiments via action retargeting, hand removal and inpainting, simulated rendering, and depth-guided compositing, coupled with a multi-stage curation pipeline ensuring data quality OOD Generalization: LIBERO-Plus 91.4% (+7.0 over π0.5), RoboTwin-C2R Hard 69.4% (+21.5 over π0.5), RoboCasa365 Composite-Unseen 14.9% (3× next best), EBench 45.6% (+18.5 over next best); RoboTwin-IF 72.0% (+22.4 over π0.5) confirming genuine language-conditioned control; 3× next best on RoboTwin-XE showing zero-shot cross-embodiment transfer Strong Real-World Performance: #1 on RoboChallenge Table30 v1 generalist track with 45% SR, sweeping top 2 and leading 3rd place by 20%; validated on real-robot platforms with 2× prior SOTA on in-domain and OOD tasks, few-shot adaptation, and cross-embodiment skill transfer

Scaling Manipulation Data#

Human-to-Robot Data Synthesis#

Robot manipulation data is scarce and expensive to collect. We introduce a Human-to-Robot synthesis pipeline that converts egocentric human manipulation videos into robot demonstrations across 15 robot embodiments via human-to-robot retargeting, hand removal and inpainting, and depth-guided robot compositing.

Data Sources#

The resulting pretraining corpus totals over 38,100 hours from three complementary sources:

Robot data (~11,420h): Open-source robotic datasets covering single-arm, dual-arm, and mobile manipulation.

Egocentric human data (~1,933h): Human manipulation videos collected from open-world environments, providing rich object-interaction and scene priors.

Human-to-Robot synthesized data (~24,808h): Generated from the egocentric data above across 15 robot platforms, serving as the primary scaling engine.

Data Curation#

We design a multi-stage curation pipeline to ensure VLA training data quality and annotation correctness. Five state-action filtering stages remove noisy actions, fix temporal misalignment, and verify kinematic consistency. Three cross-modal checks then validate that language instructions match the video content, that visual observations agree with recorded robot states, and that video frames are free of corruption.

Qwen-RobotManip Model Design#

Qwen-RobotManip couples a Qwen3.5-4B vision-language backbone with a flow-matching Diffusion Transformer (DiT) action head. Three design choices enable coherent cross-embodiment training:

Canonical State-Action Representation. All robot states and actions are mapped to a unified 80-dimensional vector covering single-arm, dual-arm, dexterous hand, and mobile base configurations. A per-dimension binary mask ensures gradients flow only through populated slots, ensuring different embodiments share the same representation without conflict.

Camera-Frame Delta Pose. End-effector actions are expressed as deltas in the camera coordinate frame rather than the robot base frame, making visually similar actions numerically proximate across embodiments. Camera extrinsics are injected via Camera Positional Encoding (CaPE) in the cross-attention layers, while intrinsics are encoded into visual tokens for field-of-view awareness. The DiT is further conditioned on end-effector type embeddings for embodiment-aware action denoising.

In-Context Policy Adaptation. The model conditions action prediction on a structured embodiment prompt (specifying robot platform, execution speed, and FPS) together with a historical observation-action chunk, enabling on-the-fly adaptation to different embodiments and behavior patterns. A stochastic context sampling strategy during training prevents action-copy shortcuts and forces genuine policy learning.

Training. Pre-training uses dual-stream co-training with a VLA stream (robot manipulation data) and a VLM stream (vision-language understanding data) at a 9:1 ratio. Post-training adopts generalist SFT on all demonstration data collected for each benchmark. We propose co-training with VL data and VLA data during post-training, which further improves OOD instruction following and generalization.

Evaluation#

Qwen-RobotManip is evaluated across 500+ simulation tasks and 80+ real-world tasks spanning various robot embodiments.

Why OOD Evaluation Matters#

A critical finding in our experiments: standard benchmarks systematically fail to capture the quality of pretraining. On in-distribution benchmarks like LIBERO and RoboTwin, models trained from scratch without any large-scale robot pretraining achieve performance comparable to previous SOTA pretrained models. Strong IID scores do not indicate genuine generalization; they can be achieved through pattern matching alone.

The separation only becomes visible under out-of-distribution evaluation: novel scenes and task variations, following unseen instructions, and cross-embodiment transfer. This is why Qwen-RobotManip adopts OOD benchmarks as the north star for evaluating robotic foundation models.

In-Distribution Results#

On standard benchmarks, Qwen-RobotManip matches or exceeds previous SOTA.

Out-of-Distribution Generalization#

Qwen-RobotManip substantially outperforms all previous models across three OOD generalization axes: task and scene variations, instruction following, and cross-embodiment transfer.

Detailed Analysis per Benchmark LIBERO-Plus — OOD robustness evaluation under 7 perturbation dimensions (camera, robot, language, lighting, background, noise, layout):

Qwen-RobotManip achieves 89.0% and Qwen-RobotManip-Context achieves 91.4% overall. The per-dimension breakdown shows that Camera and Robot perturbations benefit most from large-scale robot data pretraining, while Language and Light robustness is already provided by the VLM backbone.

RoboTwin-Clean2Rand — Models are fine-tuned on the Clean dataset and tested under progressive environmental randomizations:

Qwen-RobotManip achieves the highest Hard success rate (62.6%), retaining ~86% of its Easy performance compared to 66% for $\pi{0.5}$π 0.5​ and under 30% for models without pretraining. Qwen-RobotManip-Context further achieves 69.4% on Hard, demonstrating the effectiveness of in-context policy adaptation.

RoboCasa365 — Evaluation across atomic and long-horizon manipulation in diverse kitchen environments:

On Composite-Unseen, which requires completing long-horizon tasks in OOD scenes, Qwen-RobotManip achieves 14.9%, nearly 3× the next-best model (5.4%).

EBench — Mobile manipulation across tabletop, pick-and-place, and long-horizon tasks:

Qwen-RobotManip achieves 45.6% overall SR and a composite score of 60, outperforming $\pi{0.5}$π 0.5​ (27.1% / 41) and all other baselines by a large margin across every split.

RoboTwin-IF — Instruction following with held-out unseen instruction templates:

Qwen-RobotManip achieves 72.2% average, a +22.6 point lead over $\pi{0.5}$π 0.5​. The largest gains appear on tasks where the instruction must be parsed to select the correct action among multiple plausible alternatives, confirming genuine language-conditioned control.

Zero-Shot Cross-Embodiment — Trained on AgileX ALOHA data only, evaluated on unseen robot embodiments on our RoboTwin-XE benchmark:

Camera-frame EEF representation dramatically improves zero-shot transfer by abstracting away morphological differences. Qwen-RobotManip (eef) reaches 23.9% overall, 3.2× $\pi{0.5}$π 0.5​ (eef) at 7.5%.

Data Scaling: Alignment Enables Scale#

An important finding: only models with unified cross-embodiment representations exhibit clean log-linear data scaling behavior. Without the alignment framework (UnifiedSpace + UnifiedEEF), adding more data produces erratic or flat scaling curves. This confirms that alignment is the prerequisite for scale, not the other way around.

Real-World Experiments#

Generalization to Novel Scenes and Instructions#

In-domain evaluation across 7 tasks spanning basic pick-and-place, deformable object handling, and precision assembly:

Out-of-domain evaluation with distribution shifts in visual scenes, objects, and instructions:

Qwen-RobotManip achieves 88.6% in-domain and 87.5% OOD success, substantially outperforming $\pi{0.5}$π 0.5​ (42.9% / 37.5%) and StarVLA (20.0% / 0.0%).

Data-Efficient Skill Transfer Across Embodiments#

Few-shot adaptation. All methods are jointly finetuned on only 130 teleoperated demonstrations across 5 tasks. Qwen-RobotManip outperforms both baselines on 4 of 5 tasks:

Cross-embodiment skill transfer. A single policy jointly finetuned on 6K CobotMagic and 130 ARX demonstrations is evaluated on 4 novel tasks on ARX, for which ARX has zero training demonstrations:

The full unified framework achieves 55.0%, over 4× the best ablation variant, demonstrating that the unified representation enables skill-level transfer across kinematically different embodiments.

Complex Multi-Step Tasks and Emergent Recovery#

In the RoboChallenge Table30 v1 Generalist Track, which spans 30 tasks across 4 robot platforms, Qwen-RobotManip ranks 1st with 45% success rate and 59.83 process score, outperforming the runner-up by 20%. On 8 bimanual coordination tasks, it achieves 40% success versus $\pi{0.5}$π 0.5​’s 21.2%.

Bimanual coordination. Among the 30 benchmark tasks, 8 require tight bimanual coordination on the ALOHA platform, where the two arms must jointly stabilize, transport, and manipulate objects. Qwen-RobotManip achieves 40% average success rate, far exceeding $\pi{0.5}$π 0.5​ (21.2%), DM0 (16.2%), GR00T-MULTI (7.5%), and $\pi{0}$π 0​ (7.5%). Notably, Qwen-RobotManip is the only model to succeed on “pour fries into plate” (30% vs. 0% for all baselines), a task demanding sequential bimanual steps — stabilizing the fries box with the left arm, opening it with the right arm, picking it up, and pouring the contents onto the plate. We attribute this strong bimanual performance to two factors: (1) our pretraining corpus contains a substantial proportion of bimanual demonstration data, enabling the model to learn coordinated dual-arm control primitives; and (2) the Human-to-Robot synthesis pipeline further expands the effective bimanual pretraining data by synthesizing bimanual robot demonstrations from egocentric human videos.

Robust pick-and-place across embodiments. We identify 12 tasks across all four platforms that center on pick-and-place primitives, ranging from single-object grasping to multi-step sequential manipulation involving 4–5 objects. Qwen-RobotManip achieves 63.3% average success rate on these tasks, surpassing the next-best baseline DM0 (48.3%) by 15.0 percentage points. We attribute this capability to two factors: (1) the large-scale cross-embodiment pretraining data encodes abundant pick-and-place patterns, and (2) the unified action space enables knowledge sharing of fundamental spatial skills across different robot morphologies.

Reactive error recovery. When objects slip during grasping, the model autonomously retries until successful. This behavior emerges from pretraining at scale, not from explicit programming.

Towards Scalable Robotic Foundation Models#

Qwen-RobotManip demonstrates that the scaling recipe behind language and multimodal foundation models can be extended to robotic manipulation, but only when alignment and scale work in tandem. The unified cross-embodiment representation is what makes large-scale multi-source training productive rather than conflicting, and the Human-to-Robot synthesis pipeline provides the data diversity that alignment alone cannot supply.

Embodied intelligence is still at an early stage. Contact-rich, long-horizon real-world tasks, failure recovery, continual learning, and complex human-robot-environment interactions remain challenging. Yet Qwen-RobotManip points to a clear path forward:

Alignment unlocks scale, and scale unlocks generalization.

@article{qwenrobotmanip, title={Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models}, author={Qwen Team}, year={2026}}