Strategic Shaping of Human Prosociality: A Latent-State POMDP Framework

Zahra Zahedi; Xinyue Hu; Shashank Mehrotra; Mark Steyvers; Kumar Akash

cs.HCMar 2, 2026

Strategic Shaping of Human Prosociality: A Latent-State POMDP Framework

Authors: Zahra Zahedi, Xinyue Hu, Shashank Mehrotra, Mark Steyvers, Kumar Akash

Organizations: Honda Research Institute USA, Inc. · Department of Cognitive Science, University of California, Irvine.

Abstract

We propose a decision-theoretic framework in which a robot strategically can shape inferred human's prosocial state during repeated interactions. Modeling the human's prosociality as a latent state that evolves over time, the robot learns to infer and influence this state through its own actions, including helping and signaling. We formalize this as a latent-state POMDP with limited observations and learn the transition and observation dynamics using expectation maximization. The resulting belief-based policy balances task and social objectives, selecting actions that maximize long-term cooperative outcomes. We evaluate the model using data from user studies and show that the learned policy outperforms baseline strategies in both team performance and increasing observed human cooperative behavior.

Figures & tables

Fig. 1 : ls-POMDP model framework: the robot learns prosocial state dynamics via EM and plans actions using belief-space decision-making to strategically shape inferred human prosociality over repeated interactions.

Panel of Figure 2 — Fig. 2 : An overview of the game setup, different modes $M_{k}$ and signaling as robot action.

Fig. 3 : State reward values under various reward gradient

r

, illustrating how the reward increases with prosociality under different settings.

Fig. 4 : Policy comparison over 9 interaction steps. In The Cumulative Tokens, Never Help / Never Signal considered as baseline factor.

	Team Performance		Observation		Cumulative Tokens
	F	p-val.	F	p-val.	F	p-val.
Policy (P) ( $4$ )	$9.8314$	$0.0000^{*}$	$2.8911$	$0.0263^{*}$	$4.6720$	$0.0017^{*}$
Time (T) ( $1$ )	$50.5082$	$0.0000^{*}$	$33.1379$	$0.0000^{*}$	$1907.6132$	$0.0000^{*}$
P $\times$ T ( $4$ )	$1.1264$	$0.35$	$1.5862$	$0.18$	$5.0390$	$0.0010^{*}$

TABLE I : The results of Mixed ANOVA test

	Team Performance		Observation
Group $-$ ls-POMDP	$\Delta$	p-adj	$\Delta$	p-adj
Always help/signal	$0.027$	$1.0$	$-0.08$	$0.29$
Never help/signal	$-12.96$	$0.0^{*}$	$-0.17$	$0.0006^{*}$
Reactive	$-5.19$	$0.01^{*}$	$-0.044$	$0.84$
Myopic	$-6.13$	$0.001^{*}$	$-0.18$	$0.0003^{*}$

TABLE II : Pairwise Comparison with Tukey’s HSD.

Fig. 5 : Sensitivity of Robot Policy to Reward and Cost Parameters.

Explore similar work

Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals
Aug 5, 2026Dong Hae Mangalindan, Anand Gokhale, Francesco Bullo +1Large Language Model-Based Robot Planning Human-Robot Collaboration
Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824 · Center for Control, Dynamical Systems, and Computation, UC Santa Barbara, Santa Barbara, CA 93106 USA
Robots Influencing Humans to Reveal their Goals during Collaboration and Competition
Aug 31, 2026Debasmita Ghose, Oz Gitelson, Michal Lewkowicz +3Human-Robot Interaction Human-Robot Collaboration
Department of Computer Science, Yale University, 51 Prospect Street, New Haven, 06511, CT, USA. · Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 125 Massachusetts Ave, Cambridge, 02139, MA, USA. · Department of Computer Science, University of Colorado, Boulder, 430 UCB, 1111 Engineering Dr, Boulder, 80309, CO, USA.
Inferring World Belief States in Dynamic Real-World Environments
Apr 13, 2026Jack Kolb, Aditya Garg, Nikolai Warner +1Human-Robot Collaboration Human-Robot Interaction
College of Engineering, Georgia Institute of Technology, North Avenue, Atlanta, GA 30332, USA

Aug 5, 2026J/K move · Enter open · S save

Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals

Dong Hae Mangalindan, Anand Gokhale, Francesco Bullo, Vaibhav Srivastava

Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824 · Center for Control, Dynamical Systems, and Computation, UC Santa Barbara, Santa Barbara, CA 93106 USA

Large Language Model-Based Robot Planning Human-Robot Collaboration Human-Robot Interaction

Open paper AskPDF arXiv

We present a structured large-language-model (LLM) architecture for zero-shot human--robot coordination in a cooperative construction task with private goal views. Guided by a Dec-POMDP formulation, the architecture decomposes decision-making into (i) action-conditioned Theory-of-Mind (ToM) inference, (ii) hierarchical planning, (iii) conversation interpretation, (iv) action verification, and (v) feedback-based replanning. We compare the proposed method with an ablation without ToM inference and a multi-agent reinforcement-learning policy trained offline over many goal pairs. In human-participant experiments, the proposed method required fewer interaction steps and yielded higher post-interaction trust ratings than both baselines. These results suggest that systematically decomposing the team decision problem, using LLMs as tractable surrogates for otherwise intractable inference and planning computations, and retaining conventional verification for physical feasibility can improve both task coordination and the human experience.

Abstract

Explore similar work

Structured LLM Reasoning for Zero-Shot Human--Robot Coordination Under Hidden Goals

Robots Influencing Humans to Reveal their Goals during Collaboration and Competition

Inferring World Belief States in Dynamic Real-World Environments