Correspondence Is Not Transfer: A Pairing-Controlled Audit of Clinical Audio-Metadata Alignment
Organizations: University of Science and Technology of China, China · Independent Researcher · University of Cambridge, UK
Abstract
Clinical audio models increasingly align recordings with participant metadata or metadata-derived text. This supervision is useful but ambiguous: the paired target can contain symptoms, demographics, missingness, and cohort structure, not just disease information. We study this ambiguity by making the audio-metadata pairing itself the intervention. Our Pairing-Controlled Transfer Audit trains matched contrastive trajectories under correct pairing, global shuffling, disease-label-preserving shuffling, and label-and-recorded-sex-preserving shuffling. Each endpoint is then frozen and tested with three readouts: profile correspondence, attribute decodability, and matched disease prediction. On UKCOVID, correct improves retrieval and sex decodability over label-only shuffling, showing that the pairing intervention worked. The sex-probe contrast is largely absorbed by the sex-preserving control, while smaller retrieval differences remain. Disease-transfer contrasts, however, remain uncertain. After UKCOVID discovery fixed the pipeline, CODA TB, Cambridge COVID-19 Sounds, and Coswara were rerun as a locked external panel with the same arms, backbones, matching, readout, calibration, primary contrast, and primary metric. The external panel preserves the descriptive separation without proving a universal null. Successful profile correspondence and improved disease transfer are therefore different empirical claims.
Figures & tables
| Endpoint | Contrast | AST-6L | OPERA | HeAR |
|---|---|---|---|---|
| Sex | 19.1 [16.7,21.5] | 11.2 [9.4,13.1] | 8.6 [7.3,9.9] | |
| [ ,1.5] | 0.7 [ ,1.9] | 0.6 [ ,1.4] | ||
| 19.2 [16.8,21.4] | 10.6 [9.0,12.2] | 8.0 [6.7,9.3] | ||
| Retrieval | 0.3 [0.2,0.5] | 0.3 [0.1,0.5] | 0.7 [0.5,1.0] | |
| 0.2 [0.0,0.3] | 0.1 [ ,0.3] | 0.4 [0.2,0.6] | ||
| Disease | 0.6 [ ,2.8] | [ ,1.7] | 0.3 [ ,1.6] |
Cross-Modal Alignment
Cross-modal alignment maps representations from different data types into compatible spaces so models can compare, transfer, and combine information.
See all 440 papers