Clinical audio models are often trained by aligning recordings with participant metadata or text derived from it. This metadata is rarely a clean disease signal, since a single profile also contains symptoms, age, sex, missing responses, and recruitment information. We examine what alignment adds to frozen audio features by varying only the audio-meta data pairing. We keep the audio backbones fixed and train only a projector under correct pairing and three shuffled conditions that progressively preserve the disease label and recorded sex. We then evaluate each projector on profile retrieval, attribute probing, and disease classification on covariate-matched test sets. On UKCOVID, correct pairing clearly improves retrieval and makes sex easier to decode, confirming that the pairing manipulation works. Most of the sex-probe gap disappears when the control also preserves recorded sex, while a small retrieval gap remains. Matched disease AUROC differences are small, and every interval includes zero, so modest benefits or harms are not ruled out. The same pattern appears when we rerun the frozen pipeline on CODA TB, Cambridge COVID-19 Sounds, and Coswara, where retrieval and sex-probe contrasts are consistently positive but disease contrasts are not. We find no consistent evidence that better profile correspondence translates into better disease prediction and suggest reporting these outcomes separately.
Figures & tables
Figure 1: Pairing-controlled audit design. The intervention is the metadata donor assigned during alignment training; frozen endpoints are then evaluated through separate correspondence, decodability, and matched disease readouts.
Endpoint
Contrast
AST-6L
OPERA
HeAR
Sex
C−W
19.1 [16.7,21.5]
11.2 [9.4,13.1]
8.6 [7.3,9.9]
C−Wy,s
−0.0 [ −1.5 ,1.5]
0.7 [ −0.5 ,1.9]
0.6 [ −0.1 ,1.4]
Wy,s−W
19.2 [16.8,21.4]
10.6 [9.0,12.2]
8.0 [6.7,9.3]
Retrieval
C−W
0.3 [0.2,0.5]
0.3 [0.1,0.5]
0.7 [0.5,1.0]
C−Wy,s
0.2 [0.0,0.3]
0.1 [ −0.1 ,0.3]
0.4 [0.2,0.6]
Disease
C−W
0.6 [ −1.4 ,2.8]
−0.2 [ −1.9 ,1.7]
0.3 [ −1.1 ,1.6]
Table 1: UKCOVID paired contrasts. Values are point estimates with pointwise 95% intervals on the 0–100 scale; AUROC contrasts are percentage-point differences. Bold intervals exclude zero. OPERA denotes OPERA-CT.
Figure 2: Discovery and locked external-panel contrasts. Retrieval and sex probe panels use the C−W manipulation/decodability contrast; disease uses the prespecified clinical contrast C−Wy,s with matched-set or matching-stratum bootstrap sensitivity for the locked external panel. Colors denote frozen audio backbones (AST-6L, OPERA-CT, HeAR); x-axes use the 0–100 reporting scale. Full numeric rows are retained in the audit artifacts.
Public infant cry corpora are small, label-incompatible, and almost always evaluated one corpus at a time. We ask what this practice hides and what fixes it. Across four cry corpora screened by a multi-level leakage audit (byte-level and embedding-level deduplication plus a within-corpus train-test near-duplicate audit), we probe four frozen encoders and a handcrafted baseline under a unified five-class need ontology and shared task formulations. The audit exposes what single-corpus evaluation conceals: within-domain macro-F1 swings by 0.57-0.80 for the same encoder, cross-corpus transfer is negative on average (negative-transfer ratio 0.19-0.35, significant in 18 of 30 directed cells, BH-FDR), and 349 content-identical clip groups carry conflicting metadata labels across corpus distributions. The same audit, however, reveals a consistent way forward. Transfer into the noisiest corpus is consistently positive in effect size at matched training size and after near-duplicate removal, offering a practical recipe for small, noisy corpora. Frozen probes saturate at modest label budgets, while stabilized fine-tuning wins with full labels; domain-adaptive pretraining significantly beats stabilized fine-tuning at 5-10-shot (the 1-shot advantage is not robust to optimization-seed variance) but shows no significant advantage at 50-shot or beyond. In the tested binary, shared-label settings, ontology-mapped joint training wins in all four encoder-by-target combinations, whereas naively merging unmapped labels costs up to 37 F1 points. We release the ontology, mapping code, and audit pipeline, turning incompatible cry corpora into a usable joint-training resource.
Competitive music transcription models require large amounts of paired audio-score data, which is scarce due to collection costs, alignment difficulty, and copyright restrictions. Meanwhile, vast quantities of unpaired audio recordings and symbolic scores are freely available but have gone unused. We adopt a cycle-consistent translation framework in which a small amount of paired data acts as a minimal anchor, unlocking the full potential of the unpaired pool. We find that: unpaired data yields surprisingly large gains, especially under limited supervision; unpaired audio contributes more than unpaired scores; incorporating unlabeled audio from a new instrument during training improves transcription for that instrument without any paired supervision. Together, these results suggest that scaling unpaired data offers a practical path toward high-quality transcription for instruments where labeled data remains scarce.
Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.
Mustafa Talha İlerisoy, Hung Manh Pham, Mathias Funk +2
Eindhoven University of Technology, Eindhoven, Netherlands · Singapore Management University, Singapore