LessMimic: Long-Horizon Humanoid Interaction with Unified Distance Field Representations

Yutang Lin; Jieming Cui; Yixuan Li; Baoxiong Jia; Yixin Zhu; Siyuan Huang

cs.ROFeb 25, 2026

LessMimic: Long-Horizon Humanoid Interaction with Unified Distance Field Representations

Authors: Yutang Lin, Jieming Cui, Yixuan Li, Baoxiong Jia, Yixin Zhu, Siyuan Huang

Abstract

Humanoid robots that autonomously interact with physical environments over extended horizons represent a central goal of embodied intelligence. Existing approaches rely on reference motions or task-specific rewards, tightly coupling policies to particular object geometries and precluding multi-skill generalization within a single framework. A unified interaction representation enabling reference-free inference, geometric generalization, and long-horizon skill composition within one policy remains an open challenge. Here we show that Distance Field (DF) provides such a representation: LessMimic conditions a single whole-body policy on DF-derived geometric cues--surface distances, gradients, and velocity decompositions--removing the need for motion references, with interaction latents encoded via a Variational Auto-Encoder (VAE) and post-trained using Adversarial Interaction Priors (AIP) under Reinforcement Learning (RL). Through DAgger-style distillation that aligns DF latents with egocentric depth features, LessMimic further transfers seamlessly to vision-only deployment without motion capture (MoCap) infrastructure. A single LessMimic policy achieves 80--100% success across object scales from 0.4x to 1.6x on PickUp and SitStand where baselines degrade sharply, attains 62.1% success on 5 task instances trajectories, and remains viable up to 40 sequentially composed tasks. By grounding interaction in local geometry rather than demonstrations, LessMimic offers a scalable path toward humanoid robots that generalize, compose skills, and recover from failures in unstructured environments.

Figures & tables

(a) Shape and size generalization. The same policy lifts objects ranging from a 23 cm box to a 60 cm-diameter cylinder.

(b) Failure recovery. After perturbation, the humanoid re-initiates pickup from the new object location.

(c) Long-horizon skill composition. A single policy executes push, pick-up, carry, and sit-stand sequentially without resets.

Method	Type	Task-unified observation	No motion at inference	Long-horizon skill composition
HDMI [ 51 ]	Ref-based	✗	✗	✗
ResMimic [ 62 ]	Ref-based	✗	✗	✗
CLONE [ 26 ]	Ref-based	✗	✗	✓
Op3-Soccer [ 10 ]	Ref-free	✗	✗	✗
PhysHSI [ 50 ]	Ref-free	✗	✓	✗
Ours	Ref-free	✓	✓	✓

TABLE I: Comparison of humanoid-object interaction methods. We compare representative reference-based and reference-free methods across three axes: whether (i) a unified observation space is shared across tasks, (ii) inference requires no motion references, and (iii) a single policy supports autonomous long-horizon skill composition.

Fig. 1 : LessMimic framework overview. The policy takes as input a root trajectory command, humanoid proprioception, and a unified df -based interaction representation that captures current humanoid-object spatial- and temporal-relation. The representation is constructed from mocap or depth image and encoded into a compact latent $z_{t}$ via a vae . The policy is trained in two stages (interaction skill pre-training and discriminative post-training) and outputs actions to a whole-body controller at deployment.

Fig. 2 : Training pipeline of LessMimic . (a) Object observations from either mocap or an egocentric depth camera yield per-link df features $\mathbf{u}_{t}$ , which are velocity-decomposed and encoded into an interaction latent representation $z_{t}$ via a vae . (b) During interaction skill pre-training, a teacher policy $\pi_{\text{mimic}}$ tracks retargeted human motions to generate physically valid data, from which $\pi_{\text{base}}$ is trained via behavior cloning without access to reference motions, but takes root trajectory command $c^{\text{root}}_{t}$ , humanoid proprioception $o^{\text{prop}}$ and the df latent $z_{t}$ . (c) During discriminative post-training, $\pi_{\text{base}}$ is fine-tuned with rl guided by aip , a discriminator that regularizes interaction validity in the geometric domain across randomized object geometries, yielding $\pi_{\text{full}}$ . (d) During visual-motor distillation, $\pi_{\text{full}}$ is distilled into $\pi_{\text{vis}}$ via DAgger-style supervision, replacing mocap inputs with egocentric depth features for portable real-world deployment.

Fig. 3 : An example of the df signals during a sitting interaction. The blue curve shows the mean df distance between the humanoid and the chair across all joints, with the shaded region indicating the full range; the red curve shows the mean df gradient magnitude. As the humanoid approaches and makes contact with the chair, the distance decreases while the gradient magnitude increases, reflecting the intensifying geometric coupling. Vertical dashed lines indicate transitions between interaction phases (Stand, Sit, Sit, Stand).

Method

PickUp

R_{succ}

(

\uparrow

,%)

SitStand

R_{succ}

(

\uparrow

,%)

Push

R_{cont}

(

\uparrow

,%)

Carry

R_{succ}

(

\uparrow

,%)

0.4

0.6

1.0

1.4

1.6

0.4

1.0

1.6

0.6

1.0

1.4

Reference-based baselines

HDMI ∗

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\underline{99.7}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.5)}}

\textbf{100.0}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

40.7\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 3.3)}}

1.7\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 1.2)}}

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\underline{99.0}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

1.7\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.5)}}

0.3\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\textbf{97.3}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.5)}}

\textbf{10.6}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 2.9)}}

27.4\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.3)}}

ResMimic ∗

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

18.3\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 1.2)}}

\textbf{100.0}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\textbf{99.7}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.5)}}

63.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 10.2)}}

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\textbf{100.0}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

93.7\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.5)}}

0.4\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.2)}}

\underline{93.1}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 1.2)}}

2.2\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.7)}}

32.9\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 1.5)}}

VisualMimic

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

\textbf{100.0}\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

0.0\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}

–

20.6\,{\scriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 9.0)}}

–

Reference-free baselines

TABLE II: Comparison of baselines, ablations, and LessMimic under object-scale variation in simulation. Four interaction tasks (PickUp, SitStand, Push, Carry) are evaluated across object scales from

0.4\times

1.6\times

, with the training scale at

1.0\times

. Reference-based baselines track fixed demonstration motions; reference-free baselines and ablated variants operate without motion references. Each ablation removes one component from the full system: AIP removes the adversarial interaction prior, Syn. removes synthetic physicalization, Rand. disables geometry randomization, RL removes reinforcement learning fine-tuning, and Trans. replaces the Transformer with an mlp backbone. Results report mean

\pm

std over 3 random seeds.

R_{succ}

: task success rate;

R_{cont}

: hand contact rate over total task duration. Bold and underline denote the best and second-best results per column. ∗ : implemented by ourselves.

Panel of Figure 9 — Fig. 4 : Real-world generalization of LessMimic . (a) The policy successfully picks up a box, one of the training geometries. (b) The same policy generalizes to a soccer ball—a spherical object entirely unseen during training—demonstrating shape generalization beyond the training distribution. (c) The policy performs SitStand across two chair heights ( $12\,\mathrm{cm}$ and $46\,\mathrm{cm}$ ), maintaining stable pelvis contact across diverse seat geometries.

Method	N=5	N=10	N=15	N=25	N=40
Ablation study
Ours - AIP	$5.2\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.2)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$\underline{0.0}\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$\underline{0.0}\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$
Ours - Syn.	$\underline{22.1}\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.8)}}$	$\underline{4.9}\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.2)}}$	$\underline{1.0}\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.3)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$
Ours - Rand.	$1.9\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.1)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$
Ours - RL	$3.2\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.2)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$
Ours - Trans.	$1.7\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.1)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$	$0.0\,{\scriptscriptstyle{\color[rgb]{0.5,0.5,0.5}(\pm 0.0)}}$

TABLE III: Long-horizon skill composition with increasing task length in simulation. Success rate (%) for completing sequences of

N

randomly ordered heterogeneous tasks under a single policy without environment resets. Each ablation removes one component from the full system; see Tab. II for ablation descriptions. All ablated variants collapse to zero beyond short sequences, whereas LessMimic (Mocap) maintains non-zero success up to

N=40

. Results report mean

\pm

std over 3 random seeds. Bold denotes the best result per column.

Fig. 5 : Long-horizon skill composition in the real world. A single LessMimic policy executes a sequence of heterogeneous interaction skills without environment resets: pushing a cabinet to a target location, then picking up and carrying a box along the commanded trajectory (indicated by the yellow arrow). This demonstrates the policy’s ability to implicitly transition between distinct interaction modes under a unified df -based representation.

Method	PickUp $22cm^{3}$ ( $\uparrow$ ,%)		PickUp $60cm^{3}$ ( $\uparrow$ ,%)		SitStand $12cm$ ( $\uparrow$ ,%)		SitStand $46cm$ ( $\uparrow$ ,%)
	Real $R_{succ}$	Root $R_{acc}$	Real $R_{succ}$	Root $R_{acc}$	Real $R_{succ}$	Root $R_{acc}$	Real $R_{succ}$	Root $R_{acc}$
Ours (Mocap)	10 / 10	94.44	8 / 10	81.39	8 / 10	84.89	10 / 10	91.88
Ours (Vision)	8 / 10	89.15	7 / 10	75.24	–	–	–	–

TABLE IV: Real-world deployment of LessMimic on a physical humanoid platform. Both the mocap -based and vision-based variants are evaluated on PickUp (

22\,\mathrm{cm}^{3}

and

60\,\mathrm{cm}^{3}

) and SitStand (

12\,\mathrm{cm}

and

46\,\mathrm{cm}

seat heights) under repeated executions.

R_{succ}

reports discrete task success rate;

R_{acc}

reports root trajectory tracking accuracy, measured via external mocap for evaluation only. The vision-based variant is not evaluated on SitStand due to limited egocentric observability of back-side contacts.

Hyperparameter	Symbol	Value
Interaction Skill Pre-training
Learning rate	$\eta_{\text{pre}}$	$1\times 10^{-3}$
Batch size	$B_{\text{pre}}$	$4096\times 8$
Number of training iterations	$N_{\text{pre}}$	$24,000$
Optimizer	Adam	–
Discriminative Post-Training

TABLE A1: Training hyperparameters for each stage of the LessMimic pipeline. All three stages—interaction skill pre-training, discriminative post-training, and policy distillation—share the same policy architecture but differ in their optimization configurations. The batch size notation

M\times 8

denotes

M

samples across 8 parallel environments.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Reward Term	Formulation	Weight	Description
Task and Style Rewards
Root Tracking	$-\\|x^{\text{root}}_{t}-c^{\text{root}}_{t}\\|^{2}$	$1.0$	Follow target root trajectory without motion references.
Interaction Style	Eq. (6)	$2.0$	Discriminator-based prior on distance-field representation.
Motion Style	$1-0.25(D(s)-1)^{2}$	$1.0$	Discriminator-based motion prior for natural gait/posture.
Object Tracking	$\exp(-\\|x^{\text{obj}}_{t}-\tilde{x}^{\text{obj}}_{t}\\|^{2}/\sigma^{2})$	$1.0$	Ensures object follows the desired manipulation trajectory.
Regularization and Penalties

Appendix

TABLE A2: Reward components used in discriminative post-training. The composite reward consists of task and style rewards that drive interaction quality, and regularization penalties that ensure motion stability and physical safety. Interaction Style operates on the df -based interaction latent

z_{t}

via the aip discriminator; Motion Style operates on the full robot state via the amp discriminator to encourage natural gait and posture. Weights listed are baseline values and may be adjusted per task.

Explore similar work

DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model
Aug 23, 2026Jie Yin, Xingyu LaiHumanoid Loco-Manipulation Visuomotor Policy Learning
Independent · Tsinghua University
I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning
Oct 5, 2026Ziqi Han, Yitang Li, Junhan Sun +8Representation Learning Reinforcement Learning
Tongji University · Tsinghua University · Zhejiang University +3
InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation
Oct 5, 2026Yucheng Zhang, Sirui Xu, Jinhong Li +8Humanoid Loco-Manipulation Synthetic Data Generation for Robotics
University of Illinois Urbana-Champaign · NVIDIA

Aug 23, 2026J/K move · Enter open · S save

DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model

Jie Yin, Xingyu Lai

Independent · Tsinghua University

Humanoid Loco-Manipulation Visuomotor Policy Learning Vision-Based Robot Control Embodied World Models

Open paper AskPDF arXiv

Vision-based whole-body loco-manipulation on humanoid robots is challenging due to partial observability, contact-rich dynamics, and the difficulty of learning long-horizon behaviors from high-dimensional visual inputs. We present DreamMimic, a framework that distills privileged teacher policies into vision-based humanoid controllers via world-model-assisted distillation. Instead of using a Dreamer-style RSSM for planning, we repurpose it to learn predictive latent dynamics that serve as both a representation space and an action-conditioned multi-step supervision signal, while exposing compact predictive features to the student policy to reduce long-term drift. Beyond standard reconstruction objectives for proprioceptive and visual observations, we add auxiliary prediction heads for privileged state, contact, object state, and reward estimation. These heads provide additional supervision related to agent--object interaction and task progress, encouraging the latent representation to retain signals that are useful for contact-rich loco-manipulation. We further introduce Performance-Conditioned Guidance (PCG), a reward-driven adaptive distillation schedule that computes performance scores for both teacher and student to dynamically balance guidance and exploration. PCG prevents both premature teacher annealing and excessive teacher interference in challenging visual settings. Experiments on OMOMO and BEHAVE show improved tracking-based loco-manipulation performance over strong vision-based baselines, without exposing online privileged interaction states to the student at deployment. Qualitative simulations further examine morphology and simulator changes. These results suggest that world models can provide a useful mechanism for stabilizing visual policy distillation in contact-rich humanoid behaviors.

Abstract

Explore similar work

DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation