Computational Measurement of Team-Process Phase Dynamics in Collaborative Virtual Reality
Organizations: Data Science und Autonome Systeme Technologietransferzentrum (TTZ), 86899 Landsberg am Lech, Germany
Abstract
Collaborative virtual reality (VR) environments make team communication observable as it unfolds, but conventional transcript analyses often summarize entire trials or divide them into fixed temporal windows. Such approaches can obscure changes in team communication and coordination over time. This article presents a computational framework for detecting and interpreting dynamic team-process phases from timestamped dialogue in a collaborative VR game. The framework uses late chunking to generate context-aware transcript representations, aggregates them into temporal chunks, and applies penalized Gaussian-kernel change-point detection to identify semantic transitions in team communication. After boundary detection, term frequency--inverse document frequency (TF-IDF), non-negative matrix factorization (NMF), and representative transcript segments provide structured evidence for phase interpretation. A locally deployed large language model (LLM) uses in-context learning to generate initial interpretations that are subsequently reviewed by humans. Independently recorded interaction logs are then aligned with the detected phases to examine corresponding task-action patterns. The evaluation compares representations, pooling strategies, segmentation methods, parameter settings, reviewed phase interpretations, and phase-aligned interaction profiles. The results show that the framework identifies coherent and interpretable phase structures while preserving traceability to the underlying transcript evidence. The correspondence between transcript-derived phases and interaction behavior further supports their relevance for analyzing collaborative activity. The framework therefore offers a transparent and transferable approach for studying temporal changes in teamwork from timestamped transcripts across collaborative task settings.
Figures & tables
| Phase position | Trial 1 labels | Trial 2 labels |
|---|---|---|
| Phase 1 | Object/Resource Coordination (5/15), Player-Attribute Assignment (4/15) | Player-Attribute Assignment (6/15), Player Identity Grounding (3/15) |
| Phase 2 | Player-Attribute Assignment (4/15), Object/Resource Coordination (3/15) | Mutual Visibility Check (5/15), Player-Attribute Assignment (5/15) |
| Phase 3 | Spatial Orientation and Localization (8/15), Player-Attribute Assignment (3/15) | Mutual Visibility Check (4/14), Spatial Orientation and Localization (4/14) |
| Phase 4 | Spatial Orientation and Localization (5/12), Player-Attribute Assignment (4/12) | Spatial Orientation and Localization (4/7) |
| Phase 5 | Player Identity Grounding (1/1) | Spatial Orientation and Localization (1/2), Object/Resource Coordination (1/2) |
| Index | Phase 1 | Phase 2 | Phase 3 | Phase 4 | Phase 5 |
|---|---|---|---|---|---|
| ID01 | Player-Attribute Assignment NMF: T1: Resource identification T2: Color identity verification | Spatial Orientation and Localization NMF: T1: Numerical Sequence Coordination T2: Strategic and Empathic Approach T3: Spatial Re-orientation | Mutual Visibility Check NMF: T1: Targeted visibility checks T2: Visual perception verification | Spatial Orientation and Localization NMF: T1: Spatial Orientation via Gestures T2: Resource color identification | – |
| ID02 | Player-Attribute Assignment NMF: T1: Visual filter identification T2: Resource color identification T3: Visual Perception Calibration | Player-Attribute Assignment NMF: T1: Movement Synchronization T2: Color identity declaration | Player-Attribute Assignment NMF: T1: Avatar identification T2: Avatar visual identification | Player-Attribute Assignment NMF: T1: Resource color identification T2: Resource Color Identification | – |
| ID03 | Object / Resource Coordination NMF: T1: Task failure acknowledgment T2: Tower construction planning | Player-Attribute Assignment NMF: T1: Perceptual Color Verification T2: Error detection and confirmation T3: Spatial resource localization | Spatial Orientation and Localization NMF: T1: Purple resource identification T2: Blue Resource Identification | Spatial Orientation and Localization NMF: T1: Resource color identification T2: Resource visibility verification T3: Resource color identification | – |
| ID04 | Player-Attribute Assignment NMF: T1: Task procedure inquiry T2: Resource count verification | Object / Resource Coordination NMF: T1: Task Execution Breakdown T2: Object manipulation attempts | Spatial Orientation and Localization NMF: T1: Shared Visual Perception T2: Repeated Rejection of Incorrect Cues | Player-Attribute Assignment NMF: T1: Green Resource Identification T2: Failed resource retrieval T3: Player identification queries | Player Identity Grounding NMF: T1: Avatar color identification T2: Color identity verification T3: Directives for green guest location |
| ID05 | Action Instruction and Turn Coordination NMF: T1: Action Synchronization T2: Fragmented Attention Signals T3: Expressions of Distress T4: Personal resource identification | Player Identity Grounding NMF: T1: Execution Speed Regulation T2: Visual Grounding Struggle | Spatial Orientation and Localization NMF: T1: Resource color identification T2: Turn-taking regulation | Mutual Visibility Check NMF: T1: Visual Visibility Checks T2: Visual Grounding T3: Visual resource identification | – |
| ID06 | Player Identity Grounding NMF: T1: Identity verification T2: Expressions of Surprise T3: Collective Tower Construction | Object / Resource Coordination NMF: T1: Resource Path Selection T2: Identity verification via color | Player Identity Grounding NMF: T1: Color-based Identity Declaration T2: Mutual Visibility Establishment | – | – |
| Index | Phase 1 | Phase 2 | Phase 3 | Phase 4 | Phase 5 |
|---|---|---|---|---|---|
| ID01 | Player Identity Grounding NMF: T1: Avatar color identification T2: Identity declaration | Mutual Visibility Check NMF: T1: Visual Perception Grounding T2: Resource identification via color | Mutual Visibility Check NMF: T1: Visual confirmation of yellow resource T2: Resource location confirmation T3: Visual grounding statements | – | – |
| ID02 | Spatial Orientation and Localization NMF: T1: Target location inquiry T2: Spatial resource identification | Player-Attribute Assignment NMF: T1: Avatar Identity Verification T2: Avatar Identity Verification | Player Identity Grounding NMF: T1: Color identity confirmation T2: Visual resource identification | – | – |
| ID03 | Player-Attribute Assignment NMF: T1: Shared Visual Perception T2: Spatial reference confirmation | Player-Attribute Assignment NMF: T1: Shared Visual Perception T2: Visual Color Identification | Mutual Visibility Check NMF: T1: Visual Contact Establishment T2: Non-verbal visual signaling T3: Spatial orientation via color cues | Spatial Orientation and Localization NMF: T1: Resource color identification T2: Avatar Movement Coordination T3: Interactional Acknowledgment T4: Spatial resource identification | Spatial Orientation and Localization NMF: T1: Spatial resource orientation T2: Gestural signaling for attention T3: Red container identification |
| ID04 | Player-Attribute Assignment NMF: T1: Resource identification and confirmation T2: Resource verification T3: Visual Grounding and Orientation T4: Visual perspective sharing | Mutual Visibility Check NMF: T1: Resource Identification T2: Task execution feedback T3: Resource identification failure T4: Visual Occlusion Reporting | Mutual Visibility Check NMF: T1: Resource identification T2: Green Resource Identification T3: Visual Grounding Verification | – | – |
| ID05 | Spatial Orientation and Localization NMF: T1: pink gate identification T2: Yellow object identification | Mutual Visibility Check NMF: T1: Visual Presence Verification T2: Requests for Pauses | Mutual Visibility Check NMF: T1: Mutual Visual Grounding T2: Non-verbal gesture signaling | – | – |
| ID06 | Mutual Visibility Check NMF: T1: Resource location identification T2: Spatial Orientation and Identification T3: Visual Contact Establishment T4: Visual Grounding Confirmation | Mutual Visibility Check NMF: T1: Visual Grounding Confirmation T2: Task execution feedback | Object / Resource Coordination NMF: T1: Resource need inquiry T2: Resource identifier verification | Object / Resource Coordination NMF: T1: Resource identification T2: Identity Grounding via Name | – |
Virtual Reality
Virtual reality studies immersive, interactive environments and the technologies and human experiences that make them useful.
See all 40 papers