Robots Acquire Manipulation Skills in Seconds from a Single Human Video (HOST)

TL;DR

HOST (Human-to-robot One-Shot Skill AcquisiTion) is a framework from X Square Robot, Tsinghua, and BIT that lets a robot pick up a new manipulation skill from a single human demonstration video in an average of 29 seconds, with no task-specific fine-tuning and no parameter updates. Instead of imitating human motion, HOST runs a cascade of three self-grounded predictions: it locates the robot’s current progress on a shared task-progress manifold with the video, translates the next progression step into the robot’s own future observations, and derives actions from those predicted observations. Across 50 novel manipulation tasks it hits 62% average success — 45 percentage points above the strongest zero-shot baseline, and above a 50-demo/task fine-tuned baseline while using 50× fewer demos and being 507× faster per new skill.

Key claims

One-shot skill acquisition from a single 3rd-person human video can achieve 62% average success on the 8-task baseline suite while consuming ~29 s per skill and touching zero parameters at acquisition time [§Abstract].
HOST exceeds the zero-shot baseline by 45 percentage points and also exceeds a baseline that was fine-tuned on 50 robot demonstrations per task, at 50× fewer demos and 507× faster per-skill acquisition [§Abstract].
Mapping the robot trajectory and the human video onto a shared task-progress manifold — then redefining every training target relative to the video’s future progression — is the mechanism that lets the robot actively follow (not mimic) the demonstrated procedure across the embodiment gap [§Abstract].
The action-prediction pipeline is a cascade of self-grounded prediction: (1) estimate robot progress within the demonstrated task, (2) translate the upcoming progression into the robot’s own future observations, (3) derive actions from those predicted observations [§Abstract].
The recipe retains previously mastered skills — the acquire-in-seconds phase does not erode prior competencies, which the abstract frames as the failure mode of standard training-time loops [§Abstract].

Method

HOST decomposes one-shot human-to-robot skill acquisition into three self-grounded prediction stages chained at inference time on top of a base policy. First, a task-progress estimator locates where the robot currently sits within the demonstrated procedure by mapping both the robot trajectory and the human video onto a shared task-progress manifold; the progress coordinate — not the raw pose — is what aligns the two embodiments. Second, given the robot’s estimated progress, the model predicts the robot’s own future observations at the next progression step (rather than copying the human’s next-frame appearance), effectively translating “what happens next” into the robot’s egocentric reference frame. Third, an action head derives the actions needed to realize those predicted observations. The full pipeline is trained on targets coupled to the video demonstration — each training target is redefined so it aligns with the future progression of the video, giving the model a supervision signal about intent rather than mimicry. At deployment, the entire cascade runs without parameter updates: a new task is acquired in ~29 s by feeding in a single human demonstration video.

Results

62% average success on the 8-task benchmark used for baseline comparisons, 45 pp above the strongest parameter-update-free method [§Abstract].
Evaluated across 50 novel manipulation tasks total; the ~29 s per-skill acquisition and 62% success figures are averaged over this suite [§Abstract].
Exceeds a fine-tuned baseline that used 50 robot demonstrations per task, while requiring 50× fewer demonstrations and being 507× faster per skill [§Abstract].
Announcement thread reports the +43 pp headline number over the strongest no-parameter-update method on the 8-task benchmark (tweet- — X Square Robot Twitter post 2084144330285969476).

Why it’s interesting

HOST stakes out a distinct position on the interface-location axis tracked by Human-to-Robot Retargeting: instead of retargeting human motion (ReActor’s physics-in-loop, Do as I Do’s 4D reconstruction), the action representation (Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots‘s reduced 3-DoF), task intent (LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition), or WFM memory state (WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time), it retargets a shared task-progress coordinate and then predicts the robot’s own future observations from that coordinate. This makes it the closest sibling on the page to WAM-TTT — both keep parameters frozen at acquisition time and both push the human-robot correspondence into an inference-time computation — but HOST’s correspondence lives in a progress manifold rather than in a KV memory, and it explicitly does not need a paired human-robot meta-training stage. On the VLA Models recipe-lever board this reads as a “no-update in-context skill acquisition” recipe, contrasting with the training-time RL (π*0.6 RECAP) and clean-teleop-scaling (Spirit v1.5) positions — the 50× demo-efficiency and 507× speed-up over a 50-demo fine-tuned baseline is the load-bearing headline for the claim that inference-time cascade prediction can substitute for demo-scaling in the narrow one-shot regime.