One clip (f5730–5822, 93 frames), every stage from the raw human demo to the Cosmos reconstruction. Robot bases at the original (+0cm) placement with smoothed IK so the arms don't over-stretch.
The tracked human demo — operator with two handheld UMI grippers (each carrying a RealSense), working items between the plastic tray and the cardboard boxes. Fixed camera.
The YAM bimanual robot composited into the scene. Vive-tracked gripper poses drive 6-DoF IK per arm (j1–j6) in MuJoCo; the jaws (j7/j8) follow the recorded gripper width. Bases placed at the original calibrated spot; a two-pass IK solver ramps the one elbow reconfiguration smoothly instead of snapping.
Open-vocab YOLOE grounds the scene objects, SAM2 video-propagates each across the clip.
plastic tray
cardboard box. These are the things that must stay rigid while everything else is regenerated.
Edges of the robot overlay, masked to the robot footprint. Tuned recipe: CLAHE(4, tile 8) → blur 3 → Canny(38/0) → dilate 1 — CLAHE lifts shadow contrast so the dark wrist/gripper shows up.
Where the robot reaches into a container, its edges would overwrite the container. So:
robot edges outside the tray/box are kept,
robot edges inside are erased, and
the tray/box's own edges are added back (magenta = the protected region). (robot ∧ ¬protect) ∨ object.
The combined edge map Cosmos is locked to: fully-detailed robot + rigid tray/box geometry, no robot-edge bleed into the containers.
Cosmos Transfer2.5 locked to the step-6 control edges, operator painted out of the base image. The YAM robot does the task alone in a regenerated fulfillment-warehouse environment — boxes and tray preserved, no human, no UMI grippers. Muted-color prompt and a box-stability negative. guidance 3 · control 0.75 · sigma_max 192 · 480p · 35 steps, rendered on a Modal H100 (~70 s/clip).