Infant simulator with an embodied caregiver: Generating infant-perspective touch and vision during social interaction

Noemi Vaculinova; Matej Hoffmann

cs.ROOct 5, 2026

Infant simulator with an embodied caregiver: Generating infant-perspective touch and vision during social interaction

Authors: Noemi Vaculinova, Matej Hoffmann

Organizations: Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic

Abstract

Early development unfolds in caregiver-infant dyads, where infants' sensorimotor streams are shaped by physical contact and face-to-face interaction. Yet developmental robotics simulators commonly model infants in isolation, limiting the study of caregiver-mediated experience. We present a caregiver-enabled extension of the Multi-Modal Infant Model (MIMo) in MuJoCo that turns MIMo into a controllable platform for replaying dyadic interaction and generating dense infant-perspective observations. The system provides (i) an articulated caregiver model compatible with MIMo morphologies, parameterized from anthropometrics and optionally resized to a recorded caregiver; (ii) a workflow to replay naturalistic caregiver-infant holding and soothing interactions; and (iii) logging and visualizing the infant's first-person multimodal experience (we show touch and vision). We showcase the tool on touch by introducing origin-aware contact logging that disambiguates self-, caregiver-, and environment-generated contact and supports aggregation into touch-rate statistics comparable to manual coding. While we showcase tactile analysis, the platform is intended more broadly as a generator of multimodal dyadic datasets (touch and egocentric vision) for modeling the development of social interaction.

Figures & tables

Fig. 1 : Overview of the reanimation pipeline: the interaction is recorded using optical motion capture, then is applied to previously created caregiver and infant models, from which we obtain multimodal outputs (touch and vision) with contact logging and heatmap visualization.

Fig. 2 : All measurements needed for a custom female caregiver model based on an actor.

Fig. 3 : Models used in the experiments (left to right): newborn MIMo, two-month-old MIMo [ 24 ] , base caregiver, and actor-matched caregiver.

Fig. 4 : The areas used for touch analysis mapped on the MIMo infant model. Left to right: the MIMo model, the MIMo model segmented into areas by color and the standardized infant figure, which is used in manual annotations [ 20 ] .

Panel of Figure 5 — Fig. 5 : Representative scenarios. Each panel shows (left-to-right within the panel): reference video frame, third person simulation render, touch visualization (caregiver-origin in red; self-origin in blue), and infant left/right eye views.

Scenario	Description	Frames
baby_arms1	infant held close; caregiver sways	4000
baby_arms2	variant of baby_arms1	3978
baby_arms4	infant lifted to chest (pose change)	1466
baby_away2	infant held away; face-to-face view	2742
baby_shoulder1	infant supported on shoulder	3000
baby_shoulder2	variant of shoulder hold	1671

TABLE I : Recorded caregiver–infant interaction scenarios used for simulation replay. The interactions introduced in III-B are in bold.

Fig. 6 : The two initialization poses of the caregiver and child with an example of a real pose to be adapted. Left is the pose for a vertical child, whereas the right is for the child in a horizontal position.

Fig. 7 : The comparison of the real (blue) and simulated (orange) touches and their overlap.

Panel of Figure 9 — Fig. 8 : Scenario-level touch-rate heatmaps (touches/min). For each scenario, caregiver-origin and self-origin contacts are shown on front/back body regions.

Panel of Figure 10 — Fig. 9 : Aggregated touch-rate heatmaps (touches/min) over all replayed scenarios for two infant ages. Back and front surfaces are shown separately within each heatmap.

Fig. 10 : Comparison of touch statistics: manually coded real infant hand self-touch [ 20 ] vs. touch distributions obtainable from the simulator by filtering contact origin (hands, self, caregiver, all).

Explore similar work

Making Sense of Touch from the Child's View for Contrastive Learning
Jun 30, 2026Max Whitton, Zecheng Wang, Puchen Liu +12Tactile Sensing Visuo-Tactile Representation Learning
Boston University · University of Texas, Austin
EgoSim: Egocentric World Simulator for Embodied Interaction Generation
Apr 1, 2026Jinkun Hao, Mingda Jia, Ruiyan Wang +7HOI Generation Egocentric Vision
Shanghai Jiaotong University · Shanghai AI Laboratory · The University of Hong Kong
Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance
May 14, 2026Tianfang Zhu, Ning An, Rui Wang +4Vision-Based Tactile Sensing Cross-Modal Alignment
Institute for AI Industry Research, Tsinghua University, Beijing, China. · MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China. · School of New Media Art and Design, Beihang University, Beijing, China. +1

Jun 30, 2026, cs.LGJ/K move · Enter open · S save

Making Sense of Touch from the Child's View for Contrastive Learning

Max Whitton, Zecheng Wang, Puchen Liu, Quang Tuan Truong+11

Boston University · University of Texas, Austin

Tactile Sensing Visuo-Tactile Representation Learning

Open paper AskPDF arXiv

Is the sense of touch a mechanism for human babies' learning of visual concepts? If so, can we quantify its importance, and to what extent do babies rely on their sense of touch for visual learning? To approach these questions in a principled way, we propose a structured coding system for baby-centric touch events, yielding a dataset of 264k two-second clips of touch events coded according to this system. Using this dataset, we pretrain developmentally grounded models that reveal promising insights into the nature of baby learning from touch.

Abstract

Explore similar work

Making Sense of Touch from the Child's View for Contrastive Learning

EgoSim: Egocentric World Simulator for Embodied Interaction Generation

Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance