HiPHI

A Large-Scale Benchmarkfor High-Precision Human Motionand Object-Interaction

Jiahao Ji1,2,*, Ji Ma1,5,*, Runhan Zhang1,3,*, Runyi Yu1,4, Wenjia Wang1,5, Weiheng Chi1, Qianqian Peng1, Weichao Yan1, Yongfei Gu1, Ye Tian1, Ting Wu1, Longwei Li1, Chun Yuan3, Ruoli Dai1,†, Lei Han1,†

1Noitom Robotics2National University of Singapore3SIGS, Tsinghua University4The Hong Kong University of Science and Technology5The University of Hong Kong

*Equal contribution†Corresponding authors

Overview

The motion data foundation for physically grounded humanoid intelligence.

HiPHI turns motion capture into motion-space design. FrameNet-guided motion seeds systematically expand across direction, speed, amplitude, posture, body-part involvement, and object/contact conditions—creating broad, repeatable coverage at unprecedented scale.

617.5 h released motion 308.7 h original capture 200.1M frames 90 Hz optical MoCap 132 performers Sub-mm marker tracking accuracy

Systematic motion-space expansion

FrameNet frames and lexical units become scalable Frame–LU motion seeds instead of one-off scripts.

Mesh-level interaction grounding

Synchronized human motion, object trajectories, and object meshes preserve the full physical interaction.

Executable humanoid benchmark

Coverage, quality, tracking, scaling, and real-world G1 deployment connect data directly to humanoid learning.

Motion Atlas

The broadest kinematic motion space in the benchmark.

Explore the dataset-balanced shared embedding used to compare representative motion datasets under one unified kinematic protocol. HiPHI reaches more regions and distributes motion more broadly across them.

1620 occupied cells 1443 effective occupancy 14.1% rare-cell long-tail share 10.7% BONES-SEED rare-cell share
Motion AtlasInteractive t-SNE view with representative motion previews
Open full page

Frame–LU construction

FrameNet turns motion capture into a scalable coverage engine.

A frame defines an event type and a lexical unit identifies the word sense that evokes it. Every Frame–LU pair becomes a family of motions, expanded across path, direction, speed, rhythm, amplitude, posture, body-part involvement, support, and object/contact conditions.

22 FrameNet frames 214 Frame–LU labels 24 median actors per Frame–LU 154 Frame–LUs with ≥10 actors 53.7% top-50 duration share 46.3% long-tail duration share
Figure showing the FrameNet to HiPHI data-construction pipeline
From FrameNet to HiPHI data construction: motion-relevant frames and lexical units define motion seeds, then controlled factors expand them into a broad, structured motion space.

Loading motion units…

Representative motion preview

Select a Frame–LU chip to inspect a representative motion preview.

Construction path Frame–LU → motion-space expansion
  1. 01FrameNet frames
  2. 02Frame–LU seeds
  3. 03Controlled factors
  4. 04Optical MoCap
  5. 05BVH + synchronized object state

Physically grounded interaction

Real objects. Real constraints. Complete interaction state.

HiPHI captures full-body motion together with mesh-level object trajectories, preserving the geometry, load, inertia, resistance, and contact that shape executable humanoid behavior.

245.7 h human-object interaction 39.8% of the release 40 real-world objects 12 object categories 90 Frame–LUs 15 FrameNet frames 0.45–6.25 kg object mass range 98.1% non-conflict 95.7% near-surface grounding

Humanoid learning

From high-fidelity motion to real-world Unitree G1.

HiPHI achieves the highest matched-budget tracking success rates and fastest convergence, keeps improving as training data scales from 3 to 300 hours, and transfers to real hardware across running, sitting, crawling, carrying, flipping, and pulling.

Highest matched-budget success 3→300 h continuous scaling gains 5 runs lower failure rates Real G1 deployment

Data scaling

More HiPHI. Lower Tracking Error.

Unmirrored HiPHI training data · mean MPJPE across 10 runs · lower is better

3→300 h training scale 4 benchmarks consistent gains
HiPHI humanoid tracking scaling curves Mean MPJPE decreases as HiPHI training data increases from 3 to 300 hours across AMASS, BONES-SEED, Motion-X++, and LaFAN1.
More HiPHI training data consistently lowers cross-dataset MPJPE on AMASS, BONES-SEED, Motion-X++, and LaFAN1.

Benchmark

Broader coverage. Higher fidelity. Stronger humanoid learning.

HiPHI spans the broadest kinematic region in the benchmark, leads every reported body-motion quality metric, delivers strong human-object geometric consistency, and turns scale into measurable tracking gains.

Motion-space coverage 1620 occupied cells
1443effective occupancy
14.1%long-tail share
10.7% closest-baseline long-tail share 55×55 coverage grid HiPHI highlighted across comparisons

Best-in-benchmark body-motion quality

Lower jerk, acceleration, ground penetration, floating, and support-point drift.

Human-object geometric consistency

Massive interaction scale with strong non-conflict and near-surface grounding.

Detailed values

Compact numeric tables for exact comparison.

Release

A research-ready benchmark built to scale.

The 617.5-hour release combines high-fidelity BVH motion, synchronized object trajectories and meshes, Frame–LU indexing, natural-language descriptions, and rich metadata in one unified resource for humanoid learning.

Data format

  • BVHhuman motion
  • Object trajectoriessynchronized state
  • Object meshesinteraction geometry
  • Frame–LU indexmotion-space meaning
  • Descriptions + metadataretrieval-ready records

Metadata fields

motion_idframeluframe_luduration_secframe_countactor_idtext_annotationis_hoiobject_categoriesmirroreddatasetfpsactor_metadataobjectsobject_idobject_categorytrajectory_pathmesh_idmesh_path

License

ModalityNet Open Research License v1.0 for non-commercial scientific research, education, and evaluation.

Citation

Cite HiPHI.

If HiPHI supports your research, please cite our arXiv paper.

@article{ji2026hiphi,
  title={HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction},
  author={Ji, Jiahao and Ma, Ji and Zhang, Runhan and Yu, Runyi and Wang, Wenjia and Chi, Weiheng and Peng, Qianqian and Yan, Weichao and Gu, Yongfei and Tian, Ye and Wu, Ting and Li, Longwei and Yuan, Chun and Dai, Ruoli and Han, Lei},
  journal={arXiv preprint arXiv:2608.16222},
  year={2026}
}