Reconstruct and Retarget Human Contacts from Video for Contact‑Rich Humanoid Manipulation
Tohoku University
*Corresponding authors
Contact transitions allow humanoid robots to redistribute object support across the whole body, enabling manipulation beyond hand-only grasps. Human videos provide a scalable source of such transitions for robots to learn from. However, body occlusion disrupts object pose tracking and corrupts retargeted motion. Moreover, accurately reproducing the object trajectory does not ensure that the robot applies stabilizing forces at the intended contacts, risking slips or drops in later phases. To address these challenges, we propose ReContact, which reconstructs and retargets human-object interactions through an object-centric contact plan. The plan specifies contact phases, contact pairs, anchors, pressing directions, and sticking or sliding modes. Contact priors stabilize object pose reconstruction through occlusion. A residual policy over a frozen whole-body tracker is rewarded for making contact at planned anchors with forces aligned to the specified directions. In simulation, on a thigh-supported regrasp task, ReContact achieves the planned two-palm bottom hold in 83.5 ± 5.2% of episodes over three training seeds, versus 0.16% for the best adapted baseline. One policy trained on 247 references re-solved for new placements raises in-workspace tube-transfer success from 20.4% (source reference only) to 97.8%. On a real Unitree G1, policies execute seven contact-rich manipulation and two loco-manipulation tasks.
Side-clamp pickup, thigh-supported regrasp, turn and loaded walk, reverse regrasp and floor placement.
Six boxes from 0.7 to 1.9 kg.
Hand-held grasps filmed from the first-person view become whole-body G1 references.
The hand is retargeted to a floating RH56F1 hand, rendered from the calibrated source camera in sync with the video; a planned whole-body approach brings the 23-DoF G1 to the grasp.
The hand is retargeted to a floating RH56F1 hand, rendered from the calibrated source camera in sync with the video; a planned whole-body approach brings the 23-DoF G1 to the grasp.
The hand is retargeted to a floating RH56F1 hand, rendered from the calibrated source camera in sync with the video; a planned whole-body approach brings the 23-DoF G1 to a grasp fitted to this retarget.
Every method receives the same body and object track and runs on G1 29-DoF.

One family policy. The tube position comes from the head camera's RGB image, and the nearest family reference runs.

Deployment bottle-family policy with marker-based object localization.