Flexible Language-Guided Representation Editing for Adaptive Robot Behavior

Audrey Lee; Nathaniel Dennler; Ilia Sucholutsky; Andreea Bobu

cs.ROOct 20, 2026

Flexible Language-Guided Representation Editing for Adaptive Robot Behavior

Authors: Audrey Lee, Nathaniel Dennler, Ilia Sucholutsky, Andreea Bobu

Organizations: MIT Computer Science & Artificial Intelligence Laboratory · New York University

Abstract

Trajectory demonstrations show robots how to behave but not why, leaving them to guess which features of the task actually matter. Incorrect guesses produce misaligned internal representations and lead to incorrect robot behaviors. Because task features are often semantically defined, we argue that natural language is better suited for specifying features rather than relying on demonstrations. Although existing representation editing methods have proposed using language to specify new features, only allowing additive edits produces feature sets that are often over-specified or too narrowly scoped, especially in sequential editing tasks. We propose the Flexible Feature Editing Pipeline (FLEX), which uses language models to iteratively refine a robot's internal representation of task features from natural language corrections via three complementary operations: Add missing features, Generalize existing specific features, and Mask irrelevant features. We evaluate FLEX on 2D navigation scenarios, 7DoF Franka tabletop manipulation tasks, and a real-world study using naturalistic human language corrections and real-robot demonstrations. Comparisons against single-operation and unscaffolded baselines show that all three operations are necessary in interactive scenarios, and that FLEX achieves consistently higher normalized reward compared to other feature editing approaches while maintaining compact, transferable representations. Project available at https://mit-clear-lab.github.io/FLEX/

Figures & tables

Figure 1: Flex overview. A human’s task instruction (left) results in a misalignment between the human’s (green) and robot’s (lavender) internal task representations. To address this misalignment, Flex receives natural language corrections and applies three operations (orange) to Add missing features, Generalize overly specific features, and Mask irrelevant features. The robot then plans under the corrected feature set to optimize a reward function through demonstrations (blue), yielding trajectories that are better aligned with the human’s true reward (purple, bottom-right).

Figure 2: Simulation domains. (A) A 9x9 2D navigation environment, and (B) 7DoF Franka, a tabletop manipulation environment. Each domain includes object variants to test generalization.

Method	Add Only	Gen. Only	Mask Only	Add+Gen.	Add+Mask	Mask+Gen.	All Three
ALGAE	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{.59\pm.02}$	$\mathtt{.95\pm.03}$	$\mathtt{.77\pm.02}$	$\mathtt{.59\pm.02}$	$\mathtt{.63\pm.02}$	$\mathtt{.69\pm.03}$
MFE	$\mathtt{.95\pm.01}$	$\mathtt{.70\pm.02}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{.75\pm.02}$	$\mathtt{.43\pm.01}$	$\mathtt{.67\pm.02}$	$\mathtt{.83\pm.02}$
USF	$\mathtt{.64\pm.06}$	$\mathtt{.72\pm.03}$	$\mathtt{.90\pm.02}$	$\mathtt{.80\pm.01}$	$\mathtt{.99\pm.01}$	$\mathtt{.78\pm.04}$	$\mathtt{.78\pm.03}$
Flex (Ours)	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{.91\pm.03}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.01}}$	$\mathtt{\mathbf{.99\pm.01}}$

Table 1: Average normalized reward

\hat{R}

(mean

\pm

SE) across seven 2D Navigation scenarios. Bold indicates the best-performing method. Flex maintains strong performance across all scenarios.

Figure 3: Five-stage Franka correction sequence. At each stage, the user observes a trajectory (lavender) that is misaligned with their ground truth trajectory (green). They then provide a language correction, prompting the robot to update its internal representation $\Phi^{k}$ . Although corrections may immediately realign behavior (purple), environment changes in the subsequent stages can expose new misalignments. Stages are designed so that all three Flex operations become necessary at some point during the sequence, reflecting realistic long-term deployment conditions.

Figure 4: Average normalized reward $\hat{R}$ across five stages of the Franka correction sequences, averaged across task variants. The shaded regions indicate $\pm 1$ SE. The x-axis denotes the iteration of $\Phi^{k}$ , the feature set learned after training on Stage $k$ . Flex sustains high performance across all stages, while all baseline performances degrade as training stages accumulate.

Figure 5: Average normalized reward $\hat{R}$ of representations learned from real human language corrections and demonstrations across the three variants of 7DoF Franka correction sequences. The shaded regions indicate $\pm 1$ SE. The x-axis denotes the iteration of $\Phi^{k}$ , the feature set learned after training on Stage $k$ . Flex sustains its high performance after learning the user’s reward, while all baseline performances degrade as their feature sets accumulate spurious features.

Figure 6: Participants’ Likert ratings (

N=12

) for Flex , ALGAE , and MFE across five dimensions: task performance, intent understanding, robustness to unintended consequences, ease of use, and collaborativity. Bars show the average rating (1–7 scale) with error bars denoting

\pm 1

SE. Participants rated Flex highest on all five dimensions. Markers denote significant differences between Flex and each baseline (pairwise Wilcoxon signed-rank tests, Holm-corrected).

Operations	Initial Rep	Expected Actions
Add	goal	add stove
Generalize	goal, stove	generalize stove $\to$ heat source
Mask	goal, stove, fridge	mask fridge
Add + Generalize	goal, fridge	add stove, generalize stove $\to$ heat source
Add + Mask	goal, fridge	add stove, mask fridge
Mask + Generalize	goal, stove, fridge	mask fridge, generalize stove $\to$ heat source

Table 2: 2D Navigation misalignment scenarios. Each scenario requires a distinct subset of Flex ’s three operations. The initial features determine which operations are needed to recover the true reward, and the expected actions describe the correct response to the language correction “Don’t burn yourself on the stove.” All features are Euclidean distances to the named object or location.

Figure 7: User study interface. The side panel includes buttons to replay the ground truth (green) and current (red) trajectories, as well as pre-set camera views labeled “Top Down,” “Side,” and “Front.” Participants enter their natural language corrections in the text field on the side panel. Participants were also instructed on how to use the computer mouse to rotate, pan, and zoom the scene.

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 173 papers

Environment Edits as a Corrective Signal for Reward Learning
Oct 26, 2026Isaac Ngui, Courtney McBeth, Nancy M. AmatoRobot Learning Preference Learning
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, United States
TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations
Jun 12, 2026Kevin Robledo, Matías I. Torres Galaz, Kumar Dixhant Rai +3Robot Policy Learning Robot Policy Adaptation
Department of Computer Science California State University, Northridge · Department of Mechanical Engineering California State University, Northridge
FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model
Oct 26, 2026Qingmiao Liang, Jiatao Zhang, Runxin Zou +3Robot Task Planning Large Language Model-Based Robot Planning
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Zhejiang University, Hangzhou, China

Oct 26, 2026J/K move · Enter open · S save

Environment Edits as a Corrective Signal for Reward Learning

Isaac Ngui, Courtney McBeth, Nancy M. Amato

Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, United States

Robot Learning Preference Learning RL from Human Feedback

Open paper AskPDF arXiv

Robot corrections allow people to adjust a robot's behavior, but a single correction changes many features. Also, preferences about object attributes and relationships between objects cannot be expressed by moving the robot. We propose environment edits as a complementary form of feedback. The robot's motion is held constant as the human edits the scene, so the only explanation for the edit is the moved object. We introduce Feature Learning from Environmental and Robot-Relational Edit Traces (FERRET), which learns preferences from both robot and environment corrections. We show that environment edits are able to disambiguate features and learn preferences that robot corrections alone cannot.

Abstract

Robot Policy Adaptation

Explore similar work

Environment Edits as a Corrective Signal for Reward Learning

TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model