Flexible Language-Guided Representation Editing for Adaptive Robot Behavior

Audrey Lee; Nathaniel Dennler; Ilia Sucholutsky; Andreea Bobu

cs.ROOct 20, 2026

Flexible Language-Guided Representation Editing for Adaptive Robot Behavior

Authors: Audrey Lee, Nathaniel Dennler, Ilia Sucholutsky, Andreea Bobu

Organizations: MIT Computer Science & Artificial Intelligence Laboratory · New York University

Abstract

Trajectory demonstrations show robots how to behave but not why, leaving them to guess which features of the task actually matter. Incorrect guesses produce misaligned internal representations and lead to incorrect robot behaviors. Because task features are often semantically defined, we argue that natural language is better suited for specifying features rather than relying on demonstrations. Although existing representation editing methods have proposed using language to specify new features, only allowing additive edits produces feature sets that are often over-specified or too narrowly scoped, especially in sequential editing tasks. We propose the Flexible Feature Editing Pipeline (FLEX), which uses language models to iteratively refine a robot's internal representation of task features from natural language corrections via three complementary operations: Add missing features, Generalize existing specific features, and Mask irrelevant features. We evaluate FLEX on 2D navigation scenarios, 7DoF Franka tabletop manipulation tasks, and a real-world study using naturalistic human language corrections and real-robot demonstrations. Comparisons against single-operation and unscaffolded baselines show that all three operations are necessary in interactive scenarios, and that FLEX achieves consistently higher normalized reward compared to other feature editing approaches while maintaining compact, transferable representations. Project available at https://mit-clear-lab.github.io/FLEX/

Figures & tables

Figure 1: Flex overview. A human’s task instruction (left) results in a misalignment between the human’s (green) and robot’s (lavender) internal task representations. To address this misalignment, Flex receives natural language corrections and applies three operations (orange) to Add missing features, Generalize overly specific features, and Mask irrelevant features. The robot then plans under the corrected feature set to optimize a reward function through demonstrations (blue), yielding trajectories that are better aligned with the human’s true reward (purple, bottom-right).

Figure 2: Simulation domains. (A) A 9x9 2D navigation environment, and (B) 7DoF Franka, a tabletop manipulation environment. Each domain includes object variants to test generalization.

Method	Add Only	Gen. Only	Mask Only	Add+Gen.	Add+Mask	Mask+Gen.	All Three
ALGAE	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{.59\pm.02}$	$\mathtt{.95\pm.03}$	$\mathtt{.77\pm.02}$	$\mathtt{.59\pm.02}$	$\mathtt{.63\pm.02}$	$\mathtt{.69\pm.03}$
MFE	$\mathtt{.95\pm.01}$	$\mathtt{.70\pm.02}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{.75\pm.02}$	$\mathtt{.43\pm.01}$	$\mathtt{.67\pm.02}$	$\mathtt{.83\pm.02}$
USF	$\mathtt{.64\pm.06}$	$\mathtt{.72\pm.03}$	$\mathtt{.90\pm.02}$	$\mathtt{.80\pm.01}$	$\mathtt{.99\pm.01}$	$\mathtt{.78\pm.04}$	$\mathtt{.78\pm.03}$
Flex (Ours)	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{.91\pm.03}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.00}}$	$\mathtt{\mathbf{1.00\pm.01}}$	$\mathtt{\mathbf{.99\pm.01}}$

Table 1: Average normalized reward

\hat{R}

(mean

\pm

SE) across seven 2D Navigation scenarios. Bold indicates the best-performing method. Flex maintains strong performance across all scenarios.

Figure 3: Five-stage Franka correction sequence. At each stage, the user observes a trajectory (lavender) that is misaligned with their ground truth trajectory (green). They then provide a language correction, prompting the robot to update its internal representation $\Phi^{k}$ . Although corrections may immediately realign behavior (purple), environment changes in the subsequent stages can expose new misalignments. Stages are designed so that all three Flex operations become necessary at some point during the sequence, reflecting realistic long-term deployment conditions.

Figure 4: Average normalized reward $\hat{R}$ across five stages of the Franka correction sequences, averaged across task variants. The shaded regions indicate $\pm 1$ SE. The x-axis denotes the iteration of $\Phi^{k}$ , the feature set learned after training on Stage $k$ . Flex sustains high performance across all stages, while all baseline performances degrade as training stages accumulate.

Figure 5: Average normalized reward $\hat{R}$ of representations learned from real human language corrections and demonstrations across the three variants of 7DoF Franka correction sequences. The shaded regions indicate $\pm 1$ SE. The x-axis denotes the iteration of $\Phi^{k}$ , the feature set learned after training on Stage $k$ . Flex sustains its high performance after learning the user’s reward, while all baseline performances degrade as their feature sets accumulate spurious features.

Figure 6: Participants’ Likert ratings (

N=12

) for Flex , ALGAE , and MFE across five dimensions: task performance, intent understanding, robustness to unintended consequences, ease of use, and collaborativity. Bars show the average rating (1–7 scale) with error bars denoting

\pm 1

SE. Participants rated Flex highest on all five dimensions. Markers denote significant differences between Flex and each baseline (pairwise Wilcoxon signed-rank tests, Holm-corrected).

Operations	Initial Rep	Expected Actions
Add	goal	add stove
Generalize	goal, stove	generalize stove $\to$ heat source
Mask	goal, stove, fridge	mask fridge
Add + Generalize	goal, fridge	add stove, generalize stove $\to$ heat source
Add + Mask	goal, fridge	add stove, mask fridge
Mask + Generalize	goal, stove, fridge	mask fridge, generalize stove $\to$ heat source

Table 2: 2D Navigation misalignment scenarios. Each scenario requires a distinct subset of Flex ’s three operations. The initial features determine which operations are needed to recover the true reward, and the expected actions describe the correct response to the language correction “Don’t burn yourself on the stove.” All features are Euclidean distances to the named object or location.

Figure 7: User study interface. The side panel includes buttons to replay the ground truth (green) and current (red) trajectories, as well as pre-set camera views labeled “Top Down,” “Side,” and “Front.” Participants enter their natural language corrections in the text field on the side panel. Participants were also instructed on how to use the computer mouse to rotate, pan, and zoom the scene.

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 173 papers

Oct 26, 2026

Environment Edits as a Corrective Signal for Reward Learning

Robot corrections allow people to adjust a robot's behavior, but a single correction changes many features. Also, preferences about object attributes and relationships between objects cannot be expressed by moving the robot. We propose environment edits as a complementary form of feedback. The robot's motion is held constant as the human edits the scene, so the only explanation for the edit is the moved object. We introduce Feature Learning from Environmental and Robot-Relational Edit Traces (FERRET), which learns preferences from both robot and environment corrections. We show that environment edits are able to disambiguate features and learn preferences that robot corrections alone cannot.

Isaac Ngui, Courtney McBeth, Nancy M. Amato

Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, United States

Robot Learning Preference Learning RL from Human Feedback

Ask PDF

Jun 12, 2026

TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural language provides an intuitive way for humans to communicate such preferences. Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than language-conditioned policies.

Kevin Robledo, Matías I. Torres Galaz, Kumar Dixhant Rai +3

Department of Computer Science California State University, Northridge · Department of Mechanical Engineering California State University, Northridge

Robot Policy Learning Robot Policy Adaptation Language-Conditioned Robot Manipulation Assistive Robotics

Ask PDF

Oct 26, 2026

FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model

Large Language Models (LLMs) are widely adopted for reflective error correction in task planning; however, existing approaches primarily rely on autonomous exploration, rendering them prone to accumulating erroneous experiences during early interactions with new environments. To address this limitation, we propose the Flexible Instructional Scaffolding Reflection Framework (FISRF), an agent--mentor architecture that estimates the robot's Zone of Proximal Development (ZPD) in task planning and adaptively selects reflection strategies and mentor guidance, thereby facilitating the accumulation of effective error-correction experience. Experiments in ALFWorld and on real-world robotic platforms demonstrate that FISRF reduces the average number of error-correction rounds by 37% and redundant reasoning by 25.1%, while improving the overall task success rate by 8%. Project webpage : https://fisrfplanning.github.io/

Qingmiao Liang, Jiatao Zhang, Runxin Zou +3

Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Zhejiang University, Hangzhou, China

Robot Task Planning Large Language Model-Based Robot Planning Self-Reflection in Language Model Agents LLM Agents

Ask PDF

Abstract

Robot Policy Adaptation

Explore similar work

Environment Edits as a Corrective Signal for Reward Learning

TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model