Understanding Human-Robot Collaboration through Joint Action

Chengwen Zhang; Jianyang Wang; Chang Liu; Yukun Wang; Yuan Xu; Zhoutong Ye; Jie Cai; Chun Yu

cs.HCOct 27, 2026

Understanding Human-Robot Collaboration through Joint Action

Authors: Chengwen Zhang, Jianyang Wang, Chang Liu, Yukun Wang, Yuan Xu, Zhoutong Ye, Jie Cai, Chun Yu,

Organizations: Department of Computer Science and Technology, Tsinghua University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China ·

Abstract

Human-robot collaboration requires partners to coordinate interdependent contributions toward a shared goal. Joint Action theory provides concepts for understanding this coordination, but how these concepts are instantiated and combined across empirical research remains fragmented. We conducted a systematic review following PRISMA 2020, identifying 83 empirical papers published between 2021 and 2026. Through theory-driven framework analysis, we organized the literature into nine constructs across three capacities: Representation, Prediction, and Integration. We analyze how these constructs are distributed and co-occur across studies, revealing where empirical evidence concentrates and where it is absent. By mapping empirical HRC research onto a unified Joint Action framework, this review offers a coherent overview of HRC studies, identifies gaps in current evidence, and provides guidelines for future research.

Figures & tables

Figure 1. Understanding human–robot collaboration (HRC) through Joint Action. Overview of this systematic review: the Joint Action framework (Representation, Prediction, and Integration) organizing nine constructs, the research questions (RQs) guiding the review, the review method and scope, the resulting contributions, and six implications for future HRC research.

Figure 2. PRISMA flow diagram outlining the literature screening and inclusion process for this systematic review.

Figure 3. Overview of publication venues and years. (a) Distribution of included papers by publication venue. (b) Annual number of included papers. The 2026 data include publications available up to August 10, 2026.

Level	Primary Code	Operational Focus
Representation	JA4 Joint Attention and Shared Reference	Coordinating attention to establish a shared referent that supports subsequent joint action.
	JA5 Common Ground and Grounding	Establishing and updating mutually usable task-relevant knowledge through evidence of understanding.
	JA3 Task Co-representation	Representing one’s own task, the partner’s task, joint-task rules, and relations among contributions.
	JA2 Shared Intention and Cooperative Commitment	Supporting joint planning and execution through shared intentional structure and continued cooperative commitment.
Prediction	JA7 Task Sharing and Complementary Contribution	Dividing a joint task into interdependent roles or contributions that complement one another.
	JA6 Partner-Action Processing	Perceiving, predicting, and monitoring partner actions or joint outcomes online to support coordination.

Table 1. The final theory-derived coding framework, comprising nine multi-label primary codes organized across three levels of joint action.

Joint Action Dimension	Raw Agreement	Cohen’s $\kappa$
JA2 Shared Intention and Cooperative Commitment	77.1%	0.146
JA3 Task Co-representation	80.7%	0.614
JA4 Joint Attention and Shared Reference	81.9%	0.618
JA5 Common Ground and Grounding	86.7%	0.666
JA6 Partner-Action Processing	95.2%	0.644
JA7 Task Sharing and Complementary Contribution	91.6%	0.543

Table 2. Inter-rater reliability of paper-level Joint Action coding.

Level	Category	Papers	Broad Group	Percent
Capacity	Prediction	83	–	100.0%
Capacity	Integration	81	–	97.6%
Capacity	Representation	60	–	72.3%
JA	JA6	79	Prediction	95.2%
JA	JA7	77	Prediction	92.8%
JA	JA8	73	Integration	88.0%

Table 3. Distribution of categories.

Figure 4. Distribution of coded categories within JA2–JA10, grouped into Representation, Prediction, and Integration. Bar lengths indicate the number of papers assigned to each category; percentages use the papers coded for the corresponding JA construct as the denominator. Categories are non-exclusive, so percentages within a construct can sum to more than 100%. Horizontal bar chart of 29 categories grouped by nine Joint Action constructs and three layers. Each bar reports a paper count and its percentage within the corresponding construct.

Figure 5. Paper-level co-occurrence of coded categories within JA2–JA10. Categories are arranged by construct and grouped into Representation, Prediction, and Integration. Labels report the number of papers assigned to each category, and link widths indicate the number of papers containing both linked categories. Circular diagram connecting 29 coded categories across Representation, Prediction, and Integration. Links show category co-occurrence within papers, with wider links representing more shared papers.

Scope	Capacity Combination	Subcode 1	Subcode 2	Subcode 3	Observed	Expected	Obs. $-$ Exp.	Lift
Pair within	Integration $\times$ Integration	8.1 Signaling	8.3 Synchronization	–	35	28.92	6.08	1.21
Pair within	Prediction $\times$ Prediction	6.1 Perception	6.3 Monitoring	–	24	29.49	-5.49	0.81
Pair within	Prediction $\times$ Prediction	6.1 Perception	7.2 Leadership	–	24	17.93	6.07	1.34
Pair within	Integration $\times$ Integration	10.1 Control	10.2 Attribution	–	9	3.69	5.31	2.44
Pair within	Integration $\times$ Integration	8.3 Synchronization	9.1 Communication	–	5	10.84	-5.84	0.46
Pair cross	Prediction $\times$ Integration	6.1 Perception	8.3 Synchronization	–	36	28.92	7.08	1.25

Table 4. Filtered co-occurrence patterns among subcodes.

Scope	Capacity Combination	Subcode 1	Subcode 2	Subcode 3	$\mathbf{P(\text{Combination}\mid\text{Subcode 1})}$	$\mathbf{P(\text{Combination}\mid\text{Subcode 2})}$	$\mathbf{P(\text{Combination}\mid\text{Subcode 3})}$	Lift
Pair cross	Prediction $\times$ Integration	6.1 Perception	8.3 Synchronization	–	75.0%	72.0%	–	1.245
Pair within	Integration	8.1 Signaling	8.3 Synchronization	–	72.9%	70.0%	–	1.210
Pair within	Prediction	6.2 Prediction	6.3 Monitoring	–	66.7%	70.6%	–	1.085
Pair within	Integration	8.1 Signaling	8.2 Sequencing	–	64.6%	64.6%	–	1.117
Pair within	Prediction	6.3 Monitoring	7.1 Division	–	78.4%	64.5%	–	1.050
Pair cross	Prediction $\times$ Integration	7.1 Division	8.2 Sequencing	–	62.9%	81.3%	–	1.088

Table 5. Subcode combinations with strong conditional dependencies.

Scope	Capacity Combination	Subcode 1	Subcode 2	$\mathbf{P(\text{Combination}\mid\text{Subcode 1})}$	$\mathbf{P(\text{Combination}\mid\text{Subcode 2})}$	Dependency
Pair cross	Representation $\times$ Prediction	7.1 Division	4.2 Deictic Cues	16.1%	100.0%	4.2 Deictic Cues $\rightarrow$ 7.1 Division
Pair cross	Representation $\times$ Prediction	7.1 Division	2.3 Sustained Commitment	11.3%	100.0%	2.3 Sustained Commitment $\rightarrow$ 7.1 Division
Pair cross	Representation $\times$ Prediction	7.1 Division	5.3 Clarification	9.7%	100.0%	5.3 Clarification $\rightarrow$ 7.1 Division
Pair cross	Prediction $\times$ Integration	7.1 Division	9.1 Communication	27.4%	94.4%	9.1 Communication $\rightarrow$ 7.1 Division
Pair cross	Representation $\times$ Prediction	7.1 Division	5.2 Confirmation	25.8%	94.1%	5.2 Confirmation $\rightarrow$ 7.1 Division
Pair cross	Representation $\times$ Prediction	7.1 Division	4.3 Spatial Anchoring	17.7%	91.7%	4.3 Spatial Anchoring $\rightarrow$ 7.1 Division

Table 6. Directional dependency patterns among subcodes.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Database	Search Query	Retrieved	Included
ACM DL	$(robot^{}\ \mathrm{AND}\ human)$ $\mathrm{AND}$ $(collaborat^{}\ \mathrm{OR}\ cooperat^{}\ \mathrm{OR}\ communicat^{}\ \mathrm{OR}\ coordinat^{*})$ , searched in title, abstract, and keywords.	870	42
RO-MAN (IEEE Xplore)	Equivalent query as above, with wildcard terms expanded into corresponding word forms and searched in document title, abstract, and author keywords.	378	27
International Journal of Social Robotics (Scopus)	TITLE-ABS-KEY using the same conceptual query as above.	201	11
Science Robotics (Scopus)	Same query as above.	51	1
Computers in Human Behavior (Scopus)	Same query as above.	21	2
Nature (Scopus)	Same query as above.	2	0

Appendix

Table 7. Overview of databases, search queries, and screening results. The temporal scope of the search covers literature published between 2021 and 2026. No duplicates were identified after merging the retrieved records.

Sampling Dimension	Cues or Levels	Purpose
Author terminology	commitment/plan; attention/gaze/reference; common ground; prediction; task allocation; coordination; failure/recovery; agency	Ensure coverage of different conceptual vocabularies.
Apparent coordination structure	planned-like; emergent-like; hybrid-like; unclear	Avoid over-representing a single coordination form.
Task form	planning/allocation; assembly/handover; transport/co-manipulation; open-ended collaboration; failure recovery	Cover different forms of task interdependence.
Interaction channel	language; gaze/gesture; visual/AR; motion; haptic	Cover different interaction modalities.
Evidence type	system contribution; controlled user study; deployment or longer-term interaction	Cover different study designs.
Boundary pressure	See Table 9	Include diagnostically difficult cases for testing coding boundaries.

Appendix

Table 8. Pilot sampling dimensions and purposes.

Boundary Pair	Why Ambiguous	Pilot-Selection Trigger
Intent expression/inference vs. shared intention	Recognizing, predicting, or communicating intent does not necessarily establish shared intention.	Include studies involving intent recognition, intent communication, or joint planning.
Gaze vs. joint attention	Gaze may convey intent or social information without establishing a shared referent.	Include gaze-, pointing-, and reference-related studies.
Information display vs. common ground	Shared access to information does not necessarily establish common ground.	Include shared-display, AR-guidance, and shared-awareness studies.
Task co-representation vs. partner prediction	Representing the partner’s task differs from predicting the partner’s next action.	Include studies using task or user models for prediction or planning.
Prediction vs. monitoring	Prediction concerns future behavior, whereas monitoring concerns unfolding behavior or outcomes.	Include studies involving prediction, online adaptation, or mismatch detection.
Task sharing vs. action coordination	Task sharing concerns who contributes what; action coordination concerns how actions fit together during execution.	Include allocation, lead/follow, handover, and co-manipulation studies.

Appendix

Table 9. Boundary-pressure cases used for pilot selection.

Stage	N	Selection Logic	Diagnostic Purpose
Round 1: Anchor cases	8	Clear positive cases covering the main candidate constructs.	Establish minimum evidence thresholds and key construct distinctions.
Round 2: Construct-boundary cases	8	Papers close to neighboring constructs or prone to proxy-based false positives.	Refine inclusion, exclusion, and co-coding rules.
Round 3: Stress-test cases	8	Conceptually difficult cases involving failure/recovery, multimodality, complex coordination, or agency.	Test framework coverage in difficult cases.
Held-out codebook test	12	Selected with the same variation matrix and locked before Round 1.	Assess transferability to papers not used in codebook development.

Appendix

Table 10. Development and held-out sampling design.

Round	Year	Author	Diagnostic Purpose
R1 Anchor	2021	Vignolo et al. ( Vignolo et al., 2021 )	JA2 anchor; commitment vs. generic adaptation.
	2025	Chan et al. ( Chan et al., 2025 )	JA4/JA5 anchor; shared reference and grounding.
	2025	Sagheb et al. ( Sagheb et al., 2025 )	JA3 anchor; task/state representation vs. online processing.
	2022	Laplaza Galindo et al. ( Laplaza et al., 2022 )	JA6 anchor; future-action prediction.
	2024	Tassi et al. ( Tassi et al., 2024 )	JA7 anchor; role organization vs. action coordination.
	2025	Allen et al. ( Allen et al., 2025 )	JA8 anchor; haptic and trajectory coordination.

Appendix

Table 11. Development sample and diagnostic roles under the final codebook.

Year	Author	Boundary or Heterogeneity Covered
2026	Kim et al. ( Kim et al., 2026 )	Human-attributed failure; repair and agency.
2026	Cai et al. ( Cai et al., 2026 )	Assistive navigation; shared reference and grounding.
2026	Tulbure et al. ( Tulbure et al., 2026 )	Handover; task co-representation vs. partner-action processing.
2025	Chang et al. ( Chang et al., 2025 )	Open-ended dance; emergent coordination and agency.
2025	Pietras et al. ( Pietras et al., 2025 )	Longer-term co-adaptation; grounding and role adjustment.
2025	Bossema et al. ( Bossema et al., 2025 )	Open-ended drawing; task representation and coordination.

Appendix

Table 12. Held-out sample and the boundary or heterogeneity covered by each paper.

Code	Definition and Theory Source	Include	Exclude
JA2 Shared Intention and Cooperative Commitment	Joint planning and execution supported by shared intentional structure and continued cooperative commitment. ( Bratman, 1992 ; Bratman, 2013 )	Plan negotiation or alignment; mutual responsiveness; sustained commitment.	Intent recognition or display, simple task agreement, and task allocation alone are insufficient.
JA3 Task Co-representation	Representation of one’s own task, the partner’s task, joint-task rules, or relations among the partners’ contributions. ( Sebanz et al., 2006 ; Vesper et al., 2010 ; Vesper et al., 2017 )	Partner-task representation; joint-task models; interdependencies among subtasks.	Generic user models or prediction of the partner’s next action alone are excluded. JA3 concerns task structure; JA6 concerns online partner-action processing.
JA4 Joint Attention and Shared Reference	Coordination of attention to establish a shared object, location, or event that supports subsequent joint action. ( Sebanz et al., 2006 ; Vesper et al., 2017 )	Gaze, pointing, or reference-resolution mechanisms that establish or confirm a shared referent.	Gaze estimation, intent inference, or social signaling without a shared referent is insufficient.
JA5 Common Ground and Grounding	Activity-relevant knowledge becomes mutually usable and is established or updated through evidence of understanding. ( Clark and Brennan, 1991 ; Klein et al., 2005 )	Grounding; acknowledgement or confirmation; representational alignment; updating.	One-way information presentation or coincidentally shared information without grounding evidence is insufficient.
JA6 Partner-Action Processing	Online perception, prediction, and monitoring of partner actions or joint outcomes in support of coordination. ( Sebanz et al., 2006 ; Vesper et al., 2010 ; Vesper et al., 2017 )	Action observation; action prediction; action or outcome monitoring.	Generic perception or offline prediction disconnected from joint coordination is excluded. Prediction concerns future behavior; monitoring concerns unfolding behavior or outcomes.
JA7 Task Sharing and Complementary Contribution	Division of a joint task into interdependent roles or contributions that complement one another. ( Sebanz et al., 2006 ; Vesper et al., 2017 )	Task or role allocation; division of labor; lead/follow organization.	Independent parallel tasks and execution-level coordination alone are excluded. JA7 concerns “who does what,” whereas JA8 concerns how actions are coordinated.

Appendix

Table 13. Detailed theory-derived codebook for the nine primary Joint Action codes.

Boundary	High-Risk Error	Operational Decision
Intent expression/inference vs. shared intention	Intent recognition, prediction, or communication may be mistaken for shared intention.	Code JA2 only with evidence of joint planning, mutual responsiveness, or cooperative commitment.
Gaze vs. joint attention	Gaze may convey intent or social information without establishing a shared referent.	Code JA4 only when gaze or gesture establishes or confirms a shared referent.
Information display vs. common ground	Shared access to information may be mistaken for common ground.	Code JA5 only when the information becomes mutually usable through grounding or evidence of understanding.
Task co-representation vs. partner prediction	Representing the partner’s task may be confused with predicting the partner’s next action.	Code task structure as JA3 and future-action prediction as JA6; co-code when both are supported.
Prediction vs. monitoring	Anticipating future behavior may be confused with evaluating ongoing behavior.	Keep prediction and monitoring as distinct processes within JA6.
Task sharing vs. action coordination	Task division may be confused with execution-level coordination.	JA7 concerns “who does what”; JA8 concerns how actions are coordinated during execution.

Appendix

Table 14. High-risk discriminant boundaries used during full-text coding.

Dimension	Subdimension	Number	Papers
Shared Intention and Cooperative Commitment	Strategy Negotiation	2	( Zhong and Teearavarunyou, 2025 ; Yang and Mavrogiannis, 2025 )
	Mutual Responsiveness	3	( Zhong and Teearavarunyou, 2025 ; Vignolo et al., 2021 ; Abe et al., 2022 )
	Sustained Commitment	7	( Kim et al., 2026 ; Vermeulen et al., 2026b ; Vermeulen et al., 2026a ; Haripriyan et al., 2024 ; Eriksen and Bodenhagen, 2023 ; Yadollahi et al., 2022 ; Vignolo et al., 2021 )
Task Co-representation	Task	15	( Banerjee et al., 2026 ; Cai et al., 2026 ; Vermeulen et al., 2026a ; Tokac, 2025 ; Pathak et al., 2025 ; Pietras et al., 2025 ; Domínguez-Vidal et al., 2025 ; Deshpande et al., 2024 ; Tian et al., 2023 ; Costanzo et al., 2023 ; Silva et al., 2023 ; Guinot et al., 2022 ; Tornbjerg et al., 2021 ; Goubard and Demiris, 2025 ; Bolano et al., 2021 )
	Role	4	( Vermeulen et al., 2026b ; Miles et al., 2025 ; Domínguez-Vidal et al., 2025 ; Law et al., 2024 )
	Partner	10	( Keskin et al., 2025 ; Andriella et al., 2025 ; Angelopoulos et al., 2025 ; Sagheb et al., 2025 ; Haripriyan et al., 2024 ; Ho et al., 2024 ; Yadollahi et al., 2022 ; Norman et al., 2022 ; Kimoto et al., 2022 ; Raggioli et al., 2025 )

Appendix

Table 15. References of Joint Action Dimensions and Subdimensions.

Human-Robot Collaboration

Human-robot collaboration studies how people and robots coordinate tasks safely, effectively, and with appropriate human control.

See all 181 papers

Apr 3, 2026

Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

Fluent human--robot collaboration requires robots to continuously estimate human behaviour and anticipate future intentions. This entails reasoning jointly about continuous movements and discrete actions, which are still largely modelled in isolation. In this paper, we introduce MA-HERP, a hierarchical and recursive probabilistic framework for the joint estimation and prediction of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human--robot collaboration.

Alessandra Bulanti, Alessandro Carfì, Fulvio Mastrogiovanni

University of Genoa, Italy

Human Behavior Prediction Human Motion Prediction Human-Robot Collaboration

Ask PDF

Jun 18, 2026

Robust Assembly State Reasoning from Action Recognition for Human-Robot Collaboration

Human Action Recognition (HAR) is frequently investigated in Human-Robot Collaboration (HRC) research to understand what actions have been performed and hence the state of a collaborative task. Accurately tracking an assembly state from HAR is however not fully investigated, and in realistic scenarios is not a trivial task. This research systematically investigates and compares methods for tracking assembly state using action recognition inputs. Investigations using two diverse datasets and five state tracking approaches, including logic-based, Hidden Markov Model (HMM), and neural network (NN) methods, show that optimal approaches are not uniform across different tasks and that different methods fail under different circumstances. Testing is performed using both simulated inputs with varying noise levels and realistic inputs from a HAR model. Results show NN and HMM methods can perform well in tasks with limited variability, but for other scenarios logic-based approaches can be more robust. Methods which model expected action duration are also important for tasks with repeated actions where no additional sensing is provided.

James Fant-Male, Roel Pieters

Cognitive Robotics group, Unit of Automation Technology and Mechanical Engineering, Tampere University, 33720, Tampere, Finland

Human-Robot Collaboration Robotic Assembly

Ask PDF

Mar 4, 2026

Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport

Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. Rather than standardizing individual modules, C2C standardizes the physical semantics exchanged between them: task intent is represented by a geometrically verified payload path, partner information by a source-independent physical state, learned coordination by bounded 11-D task-space commands, and robot-specific feasibility remains inside whole-body control (WBC). The reference system combines a vision-language model with deterministic geometric verification and multi-agent reinforcement learning (MARL) for partner-aware coordination. Across nine transport scenarios, adaptive MARL variants achieve 77.1-82.1% mean success under unchanged interfaces, compared with 56.5% for a scripted-partner reference. The same contracts further support neural and interpretable hard-tree actors, two- and three-carrier teams, and simulated-to-human partner substitution. Physical Unitree G1-human tests achieve 100% success in spatially confined transport and 80% in super-long-object handling, while a three-carrier system with two G1 humanoids and one human validates the same interface structure on hardware. Together, these results show that C2C converts a tightly coupled HRC stack into a plug-compatible system in which key components can change without redefining the physical collaboration task.

Hao Zhang, Yisen Li, Ruize Geng +4

University of Texas at Arlington. · Carnegie Mellon University

Human-Robot Collaboration Multi-Agent Collaboration Humanoid Robotics Cooperative MARL

Ask PDF

Abstract

Human-Robot Collaboration

Explore similar work

Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

Robust Assembly State Reasoning from Action Recognition for Human-Robot Collaboration

Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport