From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes

Diana Cerdas-Vargas; Alejandro Romero; Richard J. Duro

cs.ROOct 26, 2026

From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes

Authors: Diana Cerdas-Vargas, Alejandro Romero, Richard J. Duro

Organizations: Integrated Group for Engineering Research (GII) at the CITIC research center, Universidade da Coruña

Abstract

This work explores a multimodal approach to the alignment of the behavior of open-ended learning autonomous robots with human purposes through the use of large language models as cultural enablers. While autonomous robots can generate their own skills and goals through interaction with the environment, they often lack mechanisms to ensure that these remain relevant and aligned to those of human users. The proposed approach makes use of the general and cultural knowledge contained in LLMs and VLMs to establish a multimodal interactive alignment process in which human purposes, expressed in natural language, are transformed into motivational components of a robot architecture in the form of robot missions and drives that guide the behavior of the robot. The system is structured into three layers for semantic alignment, mission generation, and drive modeling. This design enables a bidirectional interaction loop: human inputs define high-level purposes, robots expand them into executable missions making use of multimodal information relayed by LLMs and VLMs, and feedback maintains alignment with human intent. Experiments in simulation and with a physical robot demonstrate that the process produces valid missions and behaviors, supporting effective human-agent alignment in interactive scenarios with autonomous robots.

Figures & tables

Fig. 1 : Overview of the proposed framework. The Alignment Engine manages human–robot interaction to align human and robot purposes. The Translation Engine converts aligned purposes into missions and drives, which are integrated into the Motivational System of a cognitive architecture to guide autonomous robot behavior.

Purpose	Example of non-expert user input
Arrange apples by color	“Place the apples where they belong [as in the image]”
Classifying cylinders by color	“Sort the cylinders on the table [as in the image].”
Stacking cylinders in the center of a table from biggest to smallest	“Arrange the pieces on top of each other [as in the image]”

TABLE I : Example ambiguous user inputs for the three evaluated purposes, simulating how non-expert users describe tasks in natural language.

Arrange apples by color

“The robot should arrange the apples on the table by their color. First, the robot has to grasp the red apple (object2) and place in in the right side of the table (

x\_position>0.5

). After that, the robot has to grasp the green apple (object3) and place it on the left side of the table (

x\_position<0.5

).”

Classifying cylinders by color into baskets

“The robot must arrange the cylinders by color into baskets. First it has to reach the red cylinder (object2) with its hand, grasp it and pick it up. Then, the robot should move its hand, with the red cylinder picked, to the green basket (object4) and release it into the basket. After that, the robot should move its hand to the blue cylinder and grasp it. Then, the robot should take the blue cylinder (object3) to the position of the yellow basket (object5) and place the blue cylinder into the basket.”

Stacking cylinders in the center of the table from largest to smallest

“The robot’s task is to stack the cylinders in the center of the table by their size (based on their diameters). The largest cylinder goes at the bottom and the smallest one at the top. To do this, first, the robot should grasp the object4 (largest cylinder), move it to the center of the table. After that, the robot should grasp the object2 (second largest cylinder) and place it on top of the object4. Last, it has to grasp the object3 (smallest cylinder), move it to the center of the table and stack it on top of the object2.”

TABLE II : Detailed ideal purpose descriptions used as reference to estimate the semantic similarity.

Fig. 2 : Box plot of semantic similarity between input purposes and output descriptions generated by the Alignment Engine under multimodal and unimodal configurations, with and without human feedback.

Mission [label, weight]	Drive
[”red_apple_in_hand”, 0.2]	$\|gripper-1\|+\|x_{rob}-x_{red}\|$ $+\|y_{rob}-y_{red}\|$
[”red_apple_right_side”, 0.4]	$\max(0,\,0.5-x_{red})$
[”green_apple_in_hand”, 0.6]	$\|gripper-1\|+\|x_{rob}-x_{green}\|$ $+\|y_{rob}-y_{green}\|$
[”green_apple_left_side”, 0.8]	$\max(0,\,x_{green}-0.5)$

TABLE III : Missions and drive functions for the Arrange apples by color ideal purpose.

Fig. 3 : Validity of generated missions across the three tested purposes. The ideal purpose (Table II ) had a validity rate of 100% across the three purposes.

Fig. 4 : Validity of generated drive functions across the three tested purposes and the success rate for the ideal purposes (Table II ).

Panel of Figure 8 — Fig. 5 : Simulated and real environment used to validate the integration of the generated missions and drives within the e-MDB cognitive architecture.

Fig. 6 : Number of iterations required for the robot to achieve the assigned purpose using the Alignmnet and Translation Engines compared to using just novelty.

AI Alignment

Artificial intelligence alignment studies how to make AI systems’ goals, decisions, and actions accord with human values and legitimate constraints.

See all 126 papers

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models
Mar 22, 2026Dongyoung Kim, Sumin Park, Woomin Song +6Vision-Language-Action Models Reasoning for VLA Models
KAIST · RLWRLD · Yonsei University +1
A Multimodal Framework for Human-Multi-Agent Interaction
Mar 24, 2026Shaid Hasan, Breenice Lee, Sujan Sarker +1Human-Robot Interaction Human-AI Interaction
University of Virginia
Learning to Plan in Human-Robot Collaboration: Multimodal Reinforcement Learning for Adaptive Interaction
Sep 21, 2026Afagh Mehri Shervedani, Siyu Li, Natawut Monaikul +3Reinforcement Learning Robotic RL
University of Illinois Chicago, Chicago, IL 60607 USA · California State University Channel Islands

Mar 22, 2026J/K move · Enter open · S save

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

Dongyoung Kim, Sumin Park, Woomin Song, Seungku Kim+5

KAIST · RLWRLD · Yonsei University+1

Vision-Language-Action Models Reasoning for VLA Models Modality Gap in VLMs RL for VLMs

Open paper AskPDF arXiv

Improving embodied reasoning in multimodal-large-language models (MLLMs) is essential for building vision-language-action models (VLAs) on top of them to readily translate multimodal understanding into low-level actions. Accordingly, recent work has explored enhancing embodied reasoning in MLLMs through supervision of vision-question-answering type. However, these approaches have been reported to result in unstable VLA performance, often yielding only marginal or even negative gains. In this paper, we propose a more systematic MLLM training framework RoboAlign that reliably improves VLA performance. Our key idea is to sample action tokens via zero-shot natural language reasoning and refines this reasoning using reinforcement learning (RL) to improve action accuracy. As a result, RoboAlign bridges the modality gap between language and low-level actions in MLLMs, and facilitate knowledge transfer from MLLM to VLA. To validate the effectiveness of RoboAlign, we train VLAs by adding a diffusion-based action head on top of an MLLM backbone and evaluate them on major robotics benchmarks. Remarkably, by performing RL-based alignment after SFT using less than 1% of the data, RoboAlign achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines on LIBERO, CALVIN, and real-world environments, respectively.

Abstract

AI Alignment

Explore similar work

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

A Multimodal Framework for Human-Multi-Agent Interaction

Learning to Plan in Human-Robot Collaboration: Multimodal Reinforcement Learning for Adaptive Interaction