Lost Between Mind, Model Collaboration, and Motion: Understanding and Measuring Cross-Model Action Misalignment in Commercial Robots
Organizations: Shandong University, Jinan, China · City University of Hong Kong, Hong Kong, Hong Kong
Abstract
Embodied robots increasingly rely on large language models (LLMs) to interpret user intent and coordinate with robot-specific action models for physical interaction. This multi-model pipeline creates a critical semantic boundary across mind, model collaboration, and motion: semantic consistency may break between what the large model understands, how the small action model translates that understanding into executable commands, and what physical actions the robot ultimately performs. As a security consequence, commercial robots may execute unintended physical actions even when user inputs contain no explicit action commands. To measure this problem, we develop TARS, a black-box framework that explores the command-reasoning-action pipeline and automatically discovers and validates unintended physical behaviors. We evaluate TARS on four commercial robots across 44 experimental settings, conducting 133,400 executions. TARS uncovers widespread cross-model action misalignment and achieves up to 94% verified action coverage.
Figures & tables
| Robot | Setting | Gemini 3.1 Pro | Claude OPus 4.7 | GPT 5.5 | Llama-3.1-8B-Instruct | ||||||||||||||||
| AC | VSeed | VA | VS | TR | AC | VSeed | VA | VS | TR | AC | VSeed | VA | VS | TR | AC | VSeed | VA | VS | TR | ||
| AgiBot | |||||||||||||||||||||
| AgiBot X2 | N/A | 30.4 | 23 | 16.1 | 91.3 | 6.6 | 26.8 | 23 | 16.1 | 95.7 | 3.2 | 23.2 | 26 | 19.6 | 100.0 | 1.3 | 10.7 | 9 | 5.4 | 77.8 | 0.4 |
| Booster | |||||||||||||||||||||
| Booster K1 | Alien Weirdo | 68.4 | 67 | 57.9 | 83.6 | 8.3 | 68.4 | 50 | 42.1 | 68.0 | 7.4 | 73.7 | 74 | 68.4 | 79.7 | 9.5 | 57.9 | 50 | 47.4 | 86.0 | 8.8 |
| Booster K1 | Cute Pet | 69.6 | 58 | 39.1 | 56.9 | 99.5 | 69.6 | 67 | 39.1 | 55.2 | 100.0 | 73.9 | 76 | 52.2 | 59.2 | 99.9 | 69.6 | 21 | 43.5 | 90.5 | 99.8 |
| Robot | Setting | #Action | Translation | Intent | Hallucinated |
|---|---|---|---|---|---|
| Space | Misalign. | Misinterp. | Completion | ||
| Unitree | G1 | 23 | 3 | 6 | 142 |
| Booster K1 | Cheerful | 19 | 1 | 0 | 0 |
| Alien | 19 | 0 | 5 | 135 | |
| Tutor | 19 | 12 | 4 | 12 |
| Len. | Type | Fit | Stable | Fresh | Neutral | Adv. |
|---|---|---|---|---|---|---|
| 2 | Same | 34 | 19 | 26/34 | 21/34 | 19/34 |
| Cross | 9 | 5 | 7/9 | 7/9 | 7/9 | |
| 3 | Same | 51 | 45 | 47/51 | 46/51 | 40/51 |
| Cross | 33 | 28 | 31/33 | 25/33 | 29/33 | |
| 4 | Same | 24 | 14 | 18/24 | 20/24 | 18/24 |
| Cross | 4 | 2 | 3/4 | 3/4 | 2/4 |
| Level | Perturbation | Valid | Mean | Same action (%) | vs. P0 (pp) | Different | No action |
|---|---|---|---|---|---|---|---|
| distance | Cluster 95% CI | 95% CI | (%) | (%) | |||
| P0 | Unchanged seed | 60 | 0.000 | 71.7 [53.3, 88.3] | — | 13.3 | 15.0 |
| P1 | Surface-form edit | 59 | 0.071 | 55.9 [33.9, 78.0] | [ , ] | 15.3 | 28.8 |
| P2 | Meaning-preserving paraphrase | 60 | 0.287 | 50.0 [25.0, 73.3] | [ , ] | 15.0 | 35.0 |
| P3 | Pragmatic-frame change | 60 | 0.131 | 33.3 [15.0, 50.0] | [ , ] | 13.3 | 53.3 |
| P4 | Semantic/compositional shift | 60 | 0.699 | 11.7 [0.0, 28.3] | [ , ] | 1.7 | 86.7 |
| Category | Events | Share |
|---|---|---|
| Total cmd-triggered actions | 1,783 | 100.00% |
| Mocap-observable actions | 831 | 46.61% |
| Not observable by mocap | 952 | 53.39% |
| – Non-mocap action | 97 | 5.44% |
| – Invalid robot state | 855 | 47.95% |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Row | Emoji | Mixed | Ono. | Poem | Prose | Template | LLM |
|---|---|---|---|---|---|---|---|---|
| Early (1–1k) | Full | 22% | 20% | 18% | 19% | 18% | 2% | 59 |
| Abl. | 20% | 20% | 19% | 19% | 20% | 3% | 58 | |
| +2% | 0% | -1% | 0% | -2% | -1% | +1 | ||
| Middle (1k–2.5k) | Full | 23% | 18% | 15% | 15% | 16% | 13% | 110 |
| Abl. | 21% | 20% | 16% | 17% | 16% | 10% | 110 | |
| +2% | -2% | -1% | -2% | 0% | +3% | 0 |
| Stage | Row | Emoji | Mixed | Ono. | Poem | Prose | Template | LLM |
|---|---|---|---|---|---|---|---|---|
| Early (1–1k) | Full | 22% | 20% | 18% | 19% | 18% | 2% | 59 |
| Abl. | 19% | 21% | 19% | 22% | 18% | 1% | 54 | |
| +3% | -1% | -1% | -3% | 0% | +1% | +5 | ||
| Middle (1k–2.5k) | Full | 23% | 18% | 15% | 15% | 16% | 13% | 110 |
| Abl. | 20% | 19% | 18% | 20% | 20% | 3% | 120 | |
| +3% | -1% | -3% | -5% | -4% | +10% | -10 |
| Stage | Row | Emoji | Mixed | Ono. | Poem | Prose | Template | LLM |
|---|---|---|---|---|---|---|---|---|
| Early (1–1k) | Full | 22% | 20% | 18% | 19% | 18% | 2% | 59 |
| Abl. | 23% | 21% | 18% | 19% | 19% | 0% | 0 | |
| -1% | -1% | 0% | 0% | -1% | +2% | +59 | ||
| Middle (1k–2.5k) | Full | 23% | 18% | 15% | 15% | 16% | 13% | 110 |
| Abl. | 24% | 26% | 17% | 16% | 16% | 0% | 0 | |
| -1% | -8% | -2% | -1% | 0% | +13% | +110 |
| AgiBot X2 | Booster K1 | ||
|---|---|---|---|
| Category | Action names | Category | Action names |
| Calm | cross_arms , hands_down , head_touch , level_arm_both , level_arm_left , level_arm_right , scratch_butt , scratch_head , sleep | Control | disable_auto_greet , enable_auto_greet |
| Navigate | move_backward , move_forward , move_left , move_right | Navigate | move_backward , move_forward , move_left , move_right , turn_left , turn_left_circle , turn_right , turn_right_circle |
| Perceive | cheese , look_around , take_photos | Performance | new_year_dance , nezha_dance , stop_dance , towards_future_dance |
| Performance | cheer , clap , dance , dynamic_light_wave , sing , taiji , triple_pose , twist_left , twist_right | Social | greet , handshake , nod , shake_head , wave_hand |
| Social | blow_kiss , blow_kiss_left , bow , chest_wave_hands , chest_wave_hands_left , farewell , fist_bump , fist_bump_left , hand_heart_both , hand_heart_left , hand_heart_right , high_five_both , high_five_left , high_five_right , hug , nod , raise_hand , raise_hand_left , salute , shake_hands , shake_hands_left , shake_head , thumbs_up_both , thumbs_up_left , thumbs_up_right , victory_sign_both , victory_sign_left , victory_sign_right , wave_hands , wave_hands_left , welcome | Sound | play_angry_sound , play_happy_sound , play_sad_sound , play_scared_sound |
| Unitree G1: 42 actions | Unitree Go2: 48 actions | ||
|---|---|---|---|
| Category | Firmware-parsable action names | Category | Firmware-parsable action names |
| Agility | arm , dodge , leg , boxing_names , kick_names , defense_names | Agility | attack_people , back_flip , back_stand , front_flip , front_jump , front_punce , hand_stand , jump_left , jump_right , left_flip , roll_over |
| Calm | get_down , release_arm , rise_sit , sit_down , squat , stand_up , stop_sport | Calm | get_down , rise_sit , set_dog_height , set_robot_pose , sit_down , stand_up , stop_sport , stretch |
| Control | change_sport_mode | Navigate | avoid_obstacle , do_rotate , do_walking , free_walk , stair_walk , start_rotate , static_walk , switch_gait , walking |
| Navigate | control_navigation , do_rotate , end_auto_nav , navigate , start_auto_nav , start_rotate , walking | Perceive | do_vision_task , take_picture , update_vision_state , upload_robot_data , use_vision |
| Perceive | do_vision_task , take_picture | Performance | dance , happy_new_year , pitch , wiggle_hip |
Robotics
Robotics studies the design and control of machines that perceive, move, and act in physical environments.
See all 800 papers