Understanding Everyday Communication Patterns, Stress, and Productivity in Hybrid Work through Egocentric Wearable Audio Sensing
Authors: Tiantian Feng, Aditya Kommineni, Emily Zhou, Burcin Becerik-Gerber, Gale Lucas, Shawn C. Roll, Shrikanth Narayanan
Organizations: Signal Analysis and Interpretation Laboratory, University of Southern California, USA⋆ · Sonny Astani Department of Civil and Environmental Engineering, University of Southern California, USA‡ · Chan Division of Occupational Science and Occupational Therapy, University of Southern California, USA†
The hybrid work trend in the post-COVID period has transformed how people communicate while at work. However, there is a lack of quantitative understanding of how home and office environments shape communicative behavior and affective well-being. To study this, as a part of a larger study of the role of work environment on stress, we designed an egocentric wearable audio sensing system that captured interpersonal communication across both working locations. The system extracts real-time acoustic features on a wearable audio sensor and performs offline foreground speaker diarization to derive speaking load and activeness features. To improve diarization performance under low-dimensional feature constraints, we apply a teacher-student knowledge distillation framework that transfers knowledge from large-scale speech foundation models into a compact ResNet-based model, achieving consistent improvements in diarization performance over the baseline across three public egocentric speech datasets. In a four-month longitudinal deployment with 16 participants, our system revealed location-dependent differences in work communication patterns. Moreover, the mixed linear model analyses showed that office-based speaking load was significantly associated with higher self-reported stress and lower self-reported productivity, while speaking activeness during home-based work amplified both productivity and affective load in this cohort.
Figures & tables
Demographics
Female
Male
Total
( n=10 )
( n=6 )
( n=16 )
Age (Years)
μ±σ
42.6±8.7
40.5±7.6
43.5±7.9
Range
27-56
30-51
27-56
Self-report Work Time
Start Time (Range)
8-9AM
6-9AM
6-9AM
TABLE I: Participant demographic and work schedule characteristics. Office time (%) represented the self-reported proportion of total working time spent in the office.
Fig. 1: Overview of the study protocol. EMAs were distributed via text message four times daily, with the first and last surveys aligned to each participant’s self-reported work start and end times. Concurrently, participants wore a wearable audio sensor that performs on-device decoding of speech signals into acoustic features, followed by offline foreground speaker extraction and communicative feature computation.
Fig. 2: Overview of the FG/BG speaker diarization modeling that relies on teacher–student learning for improved FG/BG speaker diarization. Specifically, a teacher network based on a speech foundation model and a student model is first trained to perform FG/BG diarization. The teacher network is then transferred to the student network through knowledge distillation.
Model
MMCSG
ICSI-Meeting Corpus
EasyCom
VAD ↓
SC ↓
DER ↓
VAD ↓
SC ↓
DER ↓
VAD ↓
SC ↓
DER ↓
Teacher Model
WavLM-Large
7.7 ±0.2
0.7 ±0.1
8.4 ±0.2
13.0 ±0.3
0.3 ±0.1
13.3 ±0.4
24.7 ±1.4
0.7 ±0.3
25.4 ±0.5
Whisper-Tiny
14.5 ±0.5
4.0 ±0.1
18.5 ±0.5
17.5 ±0.8
1.9 ±0.3
19.4 ±0.5
31.7 ±1.5
2.2 ±0.4
33.9 ±3.0
Whisper-Small
7.1 ±0.1
0.8 ±0.1
7.9 ±0.1
10.0 ±0.2
0.2 ±0.1
10.2 ±0.2
23.1 ±0.4
1.0 ±0.1
24.1 ±0.5
Whisper-Large
6.5 ±0.2
0.8 ±0.1
7.3 ±0.3
8.3 ±0.2
0.1 ±0.1
8.4 ±0.3
21.6 ±0.6
1.0 ±0.1
22.6 ±0.5
TABLE II: Comparison of FG/BG speaker diarization results in our proposed work. VAD, SC, and DER represent voice activity detection error rate, speaker confusion rate, and diarization error rate, respectively. The lower number represents better performance.
Fig. 3: Evaluation of Distilled-ResNet models trained with and without office noise augmentation. We reported average DER across three datasets at Signal-to-noise ratios of 5, 15, and 25 dB.
Fig. 4: Overview of sensor wearing statistics.
Speaking Load
Speaking Activeness
Predictor
Coef.
p-value
Coef.
p-value
Intercept
1.59
0.912
-0.17
0.026
Sex (Male)
4.54
0.384
0.01
0.750
Location (Office)
12.61
< 0.001
0.12
< 0.001
Age
0.15
0.626
0.00
0.083
Group Var
103.13
-
0.01
-
TABLE III: Mixed linear model regression results for speaking load and activeness as dependent variables. Bold p-values indicated statistical significance ( p<0.05 ).
Fig. 5: Distribution of speaking time per hour across three workday periods by work location (home vs. office). Participants showed higher speaking time per hour when working in the office across all work periods.
Stress
Productivity
(n=1668)
(n=1793)
Predictor
Coef.
p-value
Coef.
p-value
Intercept
31.54
0.001
49.87
< 0.001
Sex (Male)
12.48
0.079
4.42
0.369
Survey (Mid-work EMA)
-2.10
0.263
0.04
0.983
Survey (End-work EMA)
-1.06
0.599
0.23
0.903
TABLE IV: Mixed linear model results examining the effects of location-specific slope term for communication patterns on stress and productivity. Bold indicates statistical significance ( p<0.05 ).
Human-Centered Artificial Intelligence Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany. · Department of Cognitive Psychology, Faculty of Psychology, Ruhr University Bochum, Bochum, Germany.