Embodiment-aware control by inference over the operator: a simulation study

Sara Falcone

cs.ROSep 29, 2026

Embodiment-aware control by inference over the operator: a simulation study

Authors: Sara Falcone

Organizations: Department of Computer Science, Seidenberg School of Computer Science and Information Systems, Pace University, New York, USA

Abstract

Teleoperation systems are tuned for channel fidelity, while whether the operator experiences the device as part of the body, the Sense of Embodiment (SoE), is measured only afterwards, by questionnaire. Predictive-processing accounts suggest controlling devices to reduce the mismatch between the operator's predictions and the returned feedback, but those predictions are unobservable, and an objective that only penalizes mismatch is minimized by removing feedback. We formulate an embodiment-aware controller, the Universal Embodiment Engine (UEE), that infers the operator's embodiment and visuo-proprioceptive cue weighting from implicit gaze and pupil signals and task outcome, and chooses bounded device settings under explicit preferences, cast as a discrete Active Inference agent. In simulations with 300 heterogeneous synthetic operators, the UEE found the suitable setting within half a minute for most operators, before identifying their exact type, and came close to an oracle in the second half of the session (embodiment 1.68 against 0.73 for the best fixed setting, on a 0-2 scale). Adapting without reading the operator did no better than fixed control, and model-free bandits did worse, whereas an expected-utility controller with the same inference did exactly as well: the benefit comes from Bayesian inference over the operator with explicit preferences, not from the information-seeking term of Active Inference. A naive prediction-error minimizer withheld feedback, as its objective implies, and lost task success (0.74 vs 0.90). The benefit shrank but persisted for operators outside the controller's model family, grew with the variety of the population, and vanished when the controller trusted an uninformative signal or when cue weighting changed mid-session without being modeled. These failures show what studies with people must establish first: calibrated signals and a model of change.

Figures & tables

Figure 1: Generative model and problem structure. a , One step of the controller’s generative model. The operator’s embodiment $E$ and cue weighting $W$ are hidden (open circles); the five observation modalities $o$ are observed (grey). The controller’s action $u_{t}$ sets the device, $S_{t+1}=u_{t}$ , and changes the operator’s embodiment dynamics. b , Long-run mean embodiment level of an operator held at each setting (rows) for each cue-weighting type (columns), under the population model. Dashed cells mark the best setting for each type (the Oracle); the outlined row is the best single setting for the population (Mechanical Transparency, MT). The two extreme types need opposite mappings, and the two mild types benefit from the extreme mapping on their side but less than the extreme types do.

Figure 2: Main results for 300 synthetic operators. A , Posterior probability of the operator’s true cue-weighting type under the UEE (mean and 95% CI; chance is 1/5). B , Embodiment level over the session (mean across operators, 5-s moving average); the shaded region is the window of the primary outcome. C , Embodiment in the last 60 s and task-success rate (means and 95% CIs), including the two model-free bandits; the expected-utility controller coincides with the UEE and is not shown. D , Embodiment in the last 60 s by the operator’s type (number of operators in brackets). Mechanical Transparency serves balanced operators well and the others poorly; the UEE comes close to the Oracle for every type.

Condition	Embodiment, last 60 s	Embodiment, session	Task success	Setting changes
Mechanical Transparency	$0.73\pm 0.68$	$0.69\pm 0.63$	$0.82\pm 0.07$	0
Yoked schedule	$0.78\pm 0.81$	$0.75\pm 0.69$	$0.82\pm 0.08$	$1.5\pm 1.0$
Naive PE minimizer	$1.35\pm 0.63$	$1.19\pm 0.53$	$0.74\pm 0.14$	$9.0\pm 3.9$
UEE	$1.68\pm 0.34$	$1.51\pm 0.31$	$0.90\pm 0.04$	$1.5\pm 1.0$
UEE without dwell	$1.68\pm 0.35$	$1.53\pm 0.32$	$0.90\pm 0.04$	$3.4\pm 2.7$
Expected-utility controller	$1.68\pm 0.34$	$1.51\pm 0.31$	$0.90\pm 0.04$	$1.5\pm 1.0$

Table 1: Outcomes by condition (300 synthetic operators, 120-s sessions; mean

\pm

SD).

		Embodiment,	Task	Time at	Mid gain,	Matched
Objective	Floor	last 60 s	success	lowest gain	last 60 s	mapping
UEE (task + embodiment)	yes	1.68	0.90	0%	0%	99%
UEE (task + embodiment)	no	1.68	0.90	0%	0%	99%
Expected utility only	yes	1.68	0.90	0%	0%	99%
Task only	yes	1.68	0.90	0%	0%	99%
Embodiment only	yes	1.68	0.90	0%	0%	99%

Table 2: Ablation of preferences and safeguards (same 300 operators; means over operators). Objective: which terms enter the choice of setting; expected utility drops the information-gain term, and information only drops the preferences, so that only the expected information gain guides the choice. Matched mapping: fraction of the last 60 s at the mapping that suits the operator.

Property changed	Scenario	Difference [95% CI]	Belief
None	matched model (reference)	$+0.92$ [ $+0.82$ , $+1.03$ ]	0.93
Noise (assumed 0.10)	0.05	$+0.93$ [ $+0.82$ , $+1.03$ ]	0.95
	0.15	$+0.92$ [ $+0.81$ , $+1.02$ ]	0.94
	0.30	$+0.89$ [ $+0.78$ , $+1.00$ ]	0.78
	0.45	$+0.85$ [ $+0.74$ , $+0.96$ ]	0.60
Pupil response to conflict	absent / doubled	$+0.92$ / $+0.92$	0.90 / 0.94

Table 3: Robustness (200 synthetic operators per scenario). Difference: UEE minus Mechanical Transparency in embodiment over the last 60 s, with 95% CI. Belief: posterior probability of the true cue-weighting type at the end of the session.

Figure 3: Where the controller fails. A , Paired difference in embodiment (last 60 s) between the UEE and Mechanical Transparency, with 95% CI, for selected scenarios (200 operators each; all scenarios in Table 3 ). Filled: CI excludes zero; open: it does not. The dotted line marks the reference. B , Embodiment over time (mean across operators, 5-s moving average) when each operator’s cue weighting is mirrored at 60 s. A model that assumes stable cue weighting keeps its first inference, falls below fixed control for about 30 s and recovers only partly by the end of the session; a small drift term lets it recover within about 35 s.

Figure 4: Population variety and operators outside the model family (200 operators per point or row). A , Embodiment in the last 60 s as the fraction of balanced operators varies, the rest split evenly over the other four types; Mechanical Transparency is recomputed as the best fixed setting for each population (the visual mapping when there are no balanced operators, the mixed mapping otherwise). B , Gain over Mechanical Transparency (mean and 95% CI) of the UEE and of the Oracle for operators whose cue weighting is continuous rather than one of five types, or whose match function differs from the one the controller assumes. The controller is unchanged throughout.

Explore similar work

Oct 6, 2026cs.RO

Beyond Task Reward: A Controller-Restriction Protocol for Evaluating Embodiment-Dependent Competence

Co-design methods optimize a robot's body and controller jointly and judge the result by one number, the task reward of the fully optimized pair. That number cannot separate morphologies whose competence depends on the controller to very different degrees. We evaluate a morphology by restricting its controller instead, recording the task competence it retains under an explicitly declared, low-complexity controller family, environment, task and search budget. On three EvoGym locomotion tasks, task reward explains only

39\%

33\%

and

10\%

of the variance in this quantity, and geometric descriptors do not predict it under run-grouped cross-validation. The measurement is reliable across optimizer restarts (ICC

(2,k) = 0.956

0.986

) but depends on the declared family: phasing the drive by actuator index instead of position ranks the same morphologies at Spearman

0.50

0.63

and reverses reward-matched pairs. As a second search objective the axis improved competence at matched task reward in

3

5

paired runs, short of a pre-registered bar of

4

. Used after an ordinary reward-only search instead, to choose within its top task-reward band, it selected a different body in all

8

runs offering a choice, at a cost of at most

0.10

reward units, and in

6

8

that body also scored higher under a held-out family. Restricted-control competence is therefore a reportable property of a co-designed morphology, interpretable only with the controller family that defines it.

Siyuan Zhang

Robotics Department, University of Michigan, Ann Arbor, MI, USA

Topic pending

Ask PDF

Jun 20, 2026cs.RO

Predictive Gaze Is Preserved but Reorganized toward Monitoring during Robot-Mediated Manipulation

Goal-directed eye movements are a fundamental component of visuomotor control, enabling humans to anticipate and guide their actions. Whether this anticipatory and task-driven behavior is preserved when actions are executed through a robot rather than through one's own body remains unclear. Here we address this question by investigating gaze behavior during goal-directed telemanipulation to determine how visuomotor control adapts to altered embodiment. Our findings show that gaze remains strongly aligned with task goals, preserving its predictive role even during robot-mediated manipulation. At the same time, teleoperation systematically redistributes visual attention toward the robotic end-effector and manipulated objects, increasing online monitoring. These findings show that predictive gaze is not lost under altered embodiment, but reorganized in response to changes in sensory feedback and control demands. More broadly, they reveal the flexibility of the human visuomotor system when the natural sensorimotor coupling is disrupted and identify gaze as an informative signal for inferring action intentions in human-robot interaction.

Manuela Uliano, Silvia Fattorini, Marco Controzzi

The BioRobotics Institute, Scuola Superiore Sant’Anna; Pisa, Italy · Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna; Pisa, Italy

Gaze Behavior Visuomotor Control Robotic Manipulation Sensorimotor Experience

Ask PDF

May 8, 2026cs.RO

Active Embodiment Identification with Reinforcement Learning for Legged Robots

We present an active embodiment identification method for legged robots that jointly learns information-seeking behavior and explicit embodiment prediction. Using a history-augmented URMA architecture, the method infers joint-level and global embodiment parameters through interaction with the environment in simulation across different morphologies.

Nico Bohlinger, Jan Peters

Department of Computer Science, Technical University of Darmstadt, Germany. · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI.

Legged Robots Embodiment Scalable Robot Learning Robot Systems Human Activity Recognition

Ask PDF