VoxCare: Studying Natural Communication Behaviors of Hospital Caregivers through Wearable Sensing of Egocentric Audio

Tiantian Feng; Kleanthis Avramidis; Anfeng Xu; Deqi Wang; Brandon M Booth; Shrikanth Narayanan

cs.SDMar 11, 2026

VoxCare: Studying Natural Communication Behaviors of Hospital Caregivers through Wearable Sensing of Egocentric Audio

Authors: Tiantian Feng, Kleanthis Avramidis, Anfeng Xu, Deqi Wang, Brandon M Booth, Shrikanth Narayanan

Organizations: University of Southern California, USA · The University of Memphis, USA

Abstract

Healthcare professionals work in complex, high-stakes environments where effective communication is critical for care delivery, team coordination, and individual well-being. However, communication activity in everyday clinical settings remains challenging to measure and largely unexplored in human behavioral research. We present VoxCare, a scalable egocentric wearable audio sensing and computing system that captures natural communication behaviors of hospital professionals in real-world settings without storing raw audio. VoxCare performs real-time, on-device acoustic feature extraction and applies a speech foundation model-guided teacher-student framework to identify foreground speech activity. From these features, VoxCare derives interpretable behavioral measures of communication frequency, duration, and vocal arousal. Our analyses reveal how, when, and how often clinicians communicate across different shifts and working units, and suggest that communication activity reflects underlying workload and stress. By enabling continuous assessment of communication patterns in everyday contexts, this study provides data-driven approaches to understand the behaviors of healthcare providers and ultimately improve healthcare delivery.

Figures & tables

Figure 1: Overview of the VoxCare . VoxCare consists of a scalable wearable audio sensor that decodes speech signals to acoustic features in real time, an offline foreground/background speaker diarization, and a behavioral feature extractor.

Figure 2: Overview of the foreground speaker diarization process. VoxCare uses a teacher–student learning framework for foreground speaker diarization. Specifically, a teacher network based on a speech foundation model is first trained to perform foreground speaker diarization. The teacher network is then transferred to a lightweight student network through knowledge distillation.

Figure 3: Example of the FG speech sessions. The x-axis represents the time in minutes. The 1st and the 2nd sessions contain the FG speech at

t={0,1}

and

t={4}

, respectively.

Figure 4: Demographics of the primary working unit in the current dataset.

	Nurse	Non-Nurse	Total
Sex
Male	49 $(19.2\%)$	26 $(10.2\%)$	$75(29.4\%)$
Female	120 $(47.1\%)$	60 $(23.5\%)$	180 $(70.6\%)$
Primary Shift
Day Shift	105 $(41.2\%)$	77 $(31.2\%)$	$182$ $(72.2\%)$
Night Shift	64 $(25.1\%)$	9 $(3.7\%)$	$73$ $(28.8\%)$

Table 1: Demographics of the primary work shift, sex and age in the current analysis.

Model	MMCSG				ICSI-Meeting Corpus
	Miss	False	Speaker	DER	Miss	False	Speaker	DER
	Detection	Alarm	Confusion	DER	Detection	Alarm	Confusion	DER
Teacher Model
Whisper-Tiny	9.8 $\pm$ 0.3	3.6 $\pm$ 0.2	3.3 $\pm$ 0.3	16.8 $\pm$ 0.3	10.5 $\pm$ 0.4	5.7 $\pm$ 0.3	1.4 $\pm$ 0.1	17.6 $\pm$ 0.1
Whisper-Small	3.6 $\pm$ 0.1	3.4 $\pm$ 0.1	0.8 $\pm$ 0.1	7.7 $\pm$ 0.2	6.0 $\pm$ 0.6	3.9 $\pm$ 0.5	0.2 $\pm$ 0.0	10.1 $\pm$ 0.1
Whisper-Large	3.5 $\pm$ 0.2	3.5 $\pm$ 0.6	0.8 $\pm$ 0.1	7.8 $\pm$ 0.5	4.3 $\pm$ 1.1	4.5 $\pm$ 0.5	0.2 $\pm$ 0.1	9.0 $\pm$ 0.6

Table 2: Comparison of foreground speaker diarization results in VoxCare .

Panel of Figure 5 — Figure 5: Violin plots comparing speaking session frequency (sessions per hour, left) and average session duration (right) for day-shift and night-shift staff.

Panel of Figure 6 — Figure 6: Violin plots comparing speaking session frequency (sessions per hour, top) and average session duration (bottom) across different hospital work units.

	90th Arousal	90th Arousal
	in 1st-half Shift	in 2nd-half Shift
Primary Work Shift
Day Shift	0.40 [0.38, 0.41]	0.41 [0.39, 0.42]
Night Shift	0.44 [0.41, 0.47]	0.35 [0.32, 0.38]
p-value (Work Shift)	0.001	$<$ 0.001
F (Work Shift)	10.34	13.78

Table 3: Comparisons of speaking arousal patterns among different work units and shifts. Mean and 95% CI were reported along with the p-value and F in the three-way ANOVA test.

Panel of Figure 7 — Figure 7: Correlations between speaking frequency and IRB scores among day-shift staff working in labs (left) and Non-ICU units (right).

Panel of Figure 8 — Figure 8: Correlations between speaking session frequency and STAI scores among day-shift (left) and night-shift (right) staff working in Non-ICU units.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Illustration of different types of diarization errors used in model evaluation.

Panel of Figure 10 — Figure 10: Correlations between 90th percentile speaking arousal and STAI scores among day-shift and night-shift staff working in Non-ICU and ICU units.

Figure 11: Changes in 90th Arousal across the shift between day shift and night shift staff working in nursing units.

Panel of Figure 12 — Figure 12: Correlations between speaking frequency and IRB scores among night-shift staff working in ICU (left) and Non-ICU units (right).

Voice Activity Detection

See all 11 papers

AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech
Oct 10, 2026Praphul Singh, Akshat Agarwal, Ratul Ghosh +5Speaker Anonymization Speech Editing
Oracle Health AI
ActivityNarrated: An Open-Ended Narrative Paradigm for Wearable Human Activity Understanding
Apr 1, 2026Lala Shakti Swarup Ray, Mengxi Liu, Alcina Pinto +4Human Activity Recognition Cross-Modal Alignment
DFKI, Germany · RPTU
Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care
May 23, 2026Vassilis Lyberatos, Edmund G. Dervakos, Eleni Adamidi +2Depression Detection Mental Health
1National Technical University of Athens, Athens, Greece · 2PsychNow

Oct 10, 2026J/K move · Enter open · S save

AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech

Praphul Singh, Akshat Agarwal, Ratul Ghosh, Aashna Kanuga+4

Oracle Health AI

Speaker Anonymization Speech Editing Clinical Speech Processing

Open paper AskPDF arXiv

Clinical speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.

Abstract

Voice Activity Detection

Explore similar work

AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech

ActivityNarrated: An Open-Ended Narrative Paradigm for Wearable Human Activity Understanding

Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care