Why did My Robot Just Change Personality? Prompting Guidelines for a Grounded Robot Persona in LLM-Based HRI
Organizations: Robotics Research Lab, RPTU Kaiserslautern, Kaiserslautern, Germany · Human-Centered Systems, Linköping University, Linköping, Sweden · Robotics Research Group, University of Hertfordshire, Hatfield, UK · Industrial Design, Eindhoven University of Technology, Eindhoven, Netherlands ·
Abstract
Large language models (LLMs) are increasingly used for verbal interaction in social robots, yet prompt design in human-robot interaction (HRI) remains underspecified. As a result, robots may present hallucinated capabilities, unclear behavioural boundaries, and misleading personas. This paper develops a framework for prompt design in LLM-based robots and introduces a structured prompt template comprising eight functional components through which robot behaviour can be specified, bounded, and adapted. The framework is grounded in a review of prior LLM-based HRI work and complemented by survey and discussion data from HRI experts gathered at the Robo-Identity workshop at IEEE RO-MAN 2025 (N=27). The qualitative findings highlight limited legibility of robot personality, the need for user adaptation, and strong ethical concerns about safety, deception, and governance. Based on these findings, we present prompting guidelines accompanied by proof-of-concept template as a structured design and reporting aid for HRI research. We argue that prompt design should be treated as a socio-technical problem rather than a minor implementation detail, requiring explicit capability boundaries, transparent behavioural assumptions, and context-sensitive safeguards to support reliable and interpretable HRI.
Figures & tables
| Reference (Sample) | Model | Social Robot | Use Case |
| Billing et al. [ 32 ] (N=N/A) | OpenAI GPT-3 | Pepper & Nao | Robot dialogue generation |
| Kim et al. [ 33 ] (N=32) | OpenAI GPT-3.5 | Pepper | LLM capability evaluation |
| Verhelst et al. [ 34 ] (N=21) | OpenAI GPT-3.5 | Furhat | Language tutoring |
| Addlesee et al. [ 35 ] (N=N/A) | Vicuna-13b-v1.5 | ARI | Multi-party HRI |
| Kim et al. [ 36 ] (N=24) | OpenAI GPT-4o-mini (fine-tuned) | piBo | Child-centric intention-aware HRI |
| Skantze et al. [ 4 ] (N=39) | OpenAI TurnGPT (fine-tuned) | Furhat | Turn-taking modelling |
| Theme | Qualitative interpretation | Illustrative quote |
| Persona as a Useful but Contested Design Abstraction | Participants saw persona as technically possible, but questioned whether it is always the right or sufficiently well-defined design direction. | “Yes — it can, the question, should it… we should discuss whether it is the right approach.” (Group 2) |
| Robot Personality Is Not Necessarily Legible | Participants doubted that users reliably infer stable personality from robot behavior, especially when cues are subtle and user interpretation varies. | “Subtle cues are often missed.” (Group 2) “subtlties are not clear, extremes are clear.” (Group 3) |
| Identity Over Time Was Framed as a Memory Problem | Longitudinal persona was framed as requiring memory, temporal ordering, and coherent state across interactions, rather than prompt style alone. | “It can support, but would be limited in memory long term due to limited context window.” (Group 2) |
| Safety and Ethics Structured the Discussion | Participants evaluated persona and identity primarily through risks of hallucination, adversarial prompting, and deceptive or unsafe outputs. | “Depends on context, deception is part of robots design.” (Group 2) “people have died from llm suggestions.” (Group 3) |
| Context-Sensitive and Vulnerability-Aware Design | Participants rejected one-size-fits-all personas and emphasised tailoring, transparency, privacy boundaries, and greater caution in vulnerable contexts. | “not a one thing fits all.” (Group 3) “Not being transparent… using the persona of a real person…” (Group 2) |
Human-Robot Interaction
Human-robot interaction studies how people and robots communicate, coordinate, and safely work or live together.
See all 452 papers