Skip to main content
Back to timeline
arXivSource publication:

Humanoids skip the robot demo grind: 500 hours of first-person human motion lifts G1 to 62.5% success on real loco-manipulation tasks

Synopsis

Humanoid whole-body loco-manipulation has depended on teleoperated robot demonstrations, which are costly and narrow in coverage. λ₀ pairs 500 hours of egocentric video with synchronized body and hand motion in the HumanVerse dataset, converts it through a human-to-humanoid alignment pipeline into G1-compatible state-action supervision, and uses it for the mid-training stage of a three-stage recipe. On a Unitree G1, the full recipe reaches 90.0% success on SIMPLE and 62.5% mean success with 80.4% weighted progress across four real-world loco-manipulation tasks.

Source-provided article image: λ_0: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
arXiv

Interpretation

Humanoid whole-body loco-manipulation can now be learned from first-person experience recorded on people, without an operator teleoperating the robot to record demonstrations for each task. Robot demonstration collection previously depended on robot availability and repeated resets, making broader task and scene coverage expensive, while first-person supervision built only from wrist and hand trajectories omitted the locomotion and posture changes that support whole-body interaction. On a Unitree G1, using only task-specific human demonstrations and no target-task robot demonstrations, mean progress is 51.7% across four whole-body manipulation tasks and 76.2% across three long-range loco-manipulation tasks, or 62.2% with all seven tasks weighted equally.

That supervision is organized into a three-stage training pipeline that first learns hand interaction, then whole-body coordination, then adapts to the target robot. Stage I pretrains on seven public egocentric hand-object datasets, training only the vision-language backbone to predict future bilateral hand-state tokens; Stage II then introduces the action expert and trains on whole-body human data converted from HumanVerse. Stage III post-trains on robot demonstrations; under fixed robot supervision, removing Stage II drops real-world success from 62.5% to 22.5%, while removing Stage I drops it to 50.0%.

The scale of human data itself yields measurable downstream gains, not only lower offline loss. Stage-I initialization, model capacity, robot demonstrations and the Stage-III update budget are held fixed while only the fraction of Stage-II human data changes. As Stage-II data goes from 0% to 5%, 10%, 25%, 50% and 100%, real-world task progress is 59.9%, 64.2%, 67.3%, 72.1%, 76.4% and 80.4%, rising monotonically for a total gain of 20.5%.

Perspective

The demonstrated scope is seven representative tasks on a Unitree G1, four whole-body manipulation and three long-range loco-manipulation tasks, plus six SIMPLE simulation tasks at Levels 0–2 with ten rollouts per task-level pair; real-world evaluation uses ten fixed initial configurations per task, and standard trials use robot-seen objects.

The 500 hours describe the recorded human corpus before conversion and validity filtering; the data-scale fits come from one run per fraction, and the authors state they describe only the observed range without establishing cross-seed reproducibility or extrapolation accuracy. The simulation summary selects the best recorded configuration per task rather than a fixed-configuration comparison, and real-world results are averaged equally across tasks without pooling training seeds or reporting seed-based confidence intervals.