Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

Payal Mohapatra; Yueyuan Sui; Haodong Yang; Benjamin Lundell; Stephen Xia; Qi Zhu

cs.LGOct 5, 2026

Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

Authors: Payal Mohapatra, Yueyuan Sui, Haodong Yang, Benjamin Lundell, Stephen Xia, Qi Zhu

Organizations: Arm Inc., USA · Northwestern University, USA

Abstract

Multimodal wearable systems must remain reliable when sensor streams become noisy or unavailable. Existing multimodal test-time adaptation (TTA) methods often assess reliability online, but cross-modal agreement can be misleading when sensors measure different physical processes, and evaluating alternative modality configurations adds inference cost. We propose CARAT, which decouples model reliance from runtime corruption detection to guide omission or attenuation, amortizing reliance estimation through source training. An asymmetric modality-dropout curriculum prepares a missingness-resilient backbone for omission and derives a frozen, backbone-specific reliance proxy from windowed input-projection gradient norms. At deployment, a lightweight one-class detector flags suspect streams, and the proxy guides a joint choice between replacing the suspect set with the backbone's trained missingness symbol and attenuating its representations before fusion, without candidate-subset evaluation. Across four wearable datasets, five corruption types, three backbones, and eight TTA baselines, CARAT achieves the highest overall macro-F1 and best mean rank (2.42), exceeding EATA, the strongest baseline, by 1.58 F1 points across 12 equally weighted dataset-backbone settings. Across five profiled configurations, CARAT uses 9.49% fewer GFLOPs and updates 47.82% fewer parameters than EATA. A pattern also emerges across sensing regimes: multimodal TTA methods such as PTA are competitive on IMU-dominated homogeneous datasets, whereas unimodal TTA methods like TENT and EATA match or exceed it on heterogeneous datasets. These results position CARAT as a practical default to wearable TTA, offering competitive robustness with modest computational requirements and benefits that vary across backbones and dataset regimes.

Figures & tables

Figure 1: Overview of CARAT . During offline training, an asymmetric modality-dropout curriculum adjusts per-modality dropout rates using windowed gradient norms

\bar{g}_{e}^{j}

, yielding a training-time reliance profile

\rho^{j}

(Section 3.2.1 ). At test time, a lightweight detector identifies suspect modalities (red; here,

x^{1}

and

x^{3}

), and the frozen reliance profile selects one intervention for the entire suspect set (Section 3.2.2 ): modality composition replaces that set with the missingness symbol

s_{0}

when the unflagged modalities retain sufficient reliance mass; otherwise, latent attenuation downweights the suspect representations through learnable scalars

a^{j}

before fusion. Adaptation updates only normalization and attenuation parameters using a sample-weighted entropy objective.

Figure 2: Per-modality gradient norm

\bar{g}_{e}^{j}

and corresponding dropout probability

p_{e}^{j}

on activity recognition (DaliaHAR) dataset for (a) wrist accelerometer and (b) wrist temperature.

Figure 3: Left: Leave-one-modality-out F1 on DaliaHAR–ShaSpec, ordered by terminal dropout probability

\rho^{j}

. Right: Mean prediction entropy and F1 under Diverse corruption (10 scenarios, 5 corruption types).

	DSADS			DaliaHAR			PAMAP2			WESAD			Overall
Method	MAE	Sha	Flex	MAE	Sha	Flex	MAE	Sha	Flex	MAE	Sha	Flex	F1 $\uparrow$	Rank $\downarrow$
Static	0.69	0.44	0.53	0.74	0.60	0.64	0.92	0.70	0.75	0.65	0.53	0.53	0.64	5.42
TENT	0.69	0.42	0.54	0.76	0.61	0.68	0.92	0.69	0.77	0.65	0.53	0.54	0.65	3.92
EATA	0.69	0.42	0.54	0.76	0.60	0.68	0.92	0.69	0.77	0.65	0.53	0.54	0.65	3.50
READ	0.69	0.39	0.54	0.74	0.60	0.65	0.91	0.66	0.75	0.64	0.53	0.53	0.64	6.83
SuMi	0.68	0.43	0.54	0.74	0.59	0.64	0.91	0.70	0.76	0.64	0.53	0.51	0.64	6.58

Table 1: Robustness across datasets and backbones. Mean macro-F1 (

\uparrow

, 0–1) over 50/50/45/95 modality-subset

\times

corruption conditions per backbone on DSADS/DaliaHAR/PAMAP2/WESAD. Overall F1 equally weights the 12 settings; Rank (

\downarrow

) averages ranks of their means before rounding, with average ranks for exact ties. Bold / underline : best / second-best displayed values; rounded ties are co-marked. MAE: MAESTRO; Sha: ShaSpec; Flex: Flex-MoE.

Figure 4: PAMAP2–MAESTRO efficiency for batches of 64, including detector overhead: GFLOPs, updated parameters, latency, and CPU energy. Full measurements: Appendix Table 25 .

Figure 5: Absolute

\Delta

F1 relative to Static on MAESTRO, averaged across the evaluated modality-subset and corruption conditions. Full results: Appendix Table 28 .

ID	Variant	w_acc	w_temp	all_w	Div.
R1	Always drop	0.63	0.70	0.61	0.58
R2	Always attenuate	0.70	0.73	0.66	0.63
R3	Noisy flags	0.67	0.73	0.65	0.63
R4	No detector	0.66	0.75	0.60	0.61
R5	Oracle flags	0.70	0.75	0.68	0.63
R6	No attenuation	0.67	0.74	0.66	0.62

Table 2: Component ablations. Macro-F1 (

\uparrow

); all_w : all_wrist . R8 disables online updates; R9 changes source training and its prior. Bold / underline : best/second displayed score, including ties.

TTA	OGM	OPM	MIAM
Static	0.57	0.59	0.58
EATA	0.62	0.63	0.61
CARAT	0.63	0.65	0.63

Table 3: TTA across source-training regimes. DaliaHAR–ShaSpec, Div. macro-F1 (

\uparrow

Prior	w_acc	w_temp	all_w	Div.
Rev.	0.64	0.71	0.62	0.61
Rand.	0.67	0.73	0.65	0.63
Jit.	0.70	0.75	0.67	0.63
CARAT	0.70	0.75	0.67	0.63

Table 4: Prior perturbations. Rev.: reversed ordering; Rand.: random; Jit.: order-preserving jitter.

Figure 6: Detector-quality bins on Div.

A. Noise	Max. $\rho$ (noised)		F1 pts.
None	–		80.20 ±1.40
T	0.01		81.40
T, E	0.00		82.50
T, E, B	0.00		80.40
B. Width	$100r^{\mathrm{C}}$	$100r^{\mathrm{T}}$	F1 pts.
None	23.70 ±13.00	9.80 ±4.10	80.30 ±0.70

Table 5: DaliaHAR–MAESTRO diagnostics. T/E/B: wrist temp./EDA/BVP; C: chest acc.

\rho

: terminal dropout;

r

: normalized reliance.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Corruption	Default parameters
Discrete missingness	$\rho=0.05$
Waveform anomaly	$\rho=0.05$ , $r=5$ , $c=3.0$ , $L=5$ , $f=0.04$ , $a=2.0$
Nonlinear degradation	$q=60$ , $\beta=0.3$
Frequency attenuation	$f_{c}=0.3\cdot f_{s}/2$ , $A=20$ dB
Nonstationary noise	$\mathrm{SNR}_{0}=30$ dB, $\mathrm{SNR}_{T}=5$ dB

Appendix

Table 6: Default parameters for each test-time corruption.

Figure 7: Test-time noise injections across heterogeneous wearable sensors. Rows show four modalities — wrist_ACC (motion), wrist_BVP (photoplethysmography), chest_ECG (electrocardiogram), and chest_EDA (electrodermal activity). Columns show the five noise models from Appendix A : discrete missingness (sparse zero replacements), waveform anomaly (segment substitution by a square wave or sinusoid), nonlinear sensor degradation (amplitude clipping with power-law compression), frequency-selective attenuation (high-frequency rolloff), and nonstationary noise (linearly increasing noise variance). Orange highlights mark localized perturbations; shaded panels indicate window-wide corruption. The same mechanism produces qualitatively different distortions across modalities.

Panel of Figure 9 — Figure 9: Detector quality (F1 at flagging suspect modalities) across (modality, noise) combinations on each dataset.

			Accuracy			F1
Dataset	Backbone	Variant	Complete	20% miss.	40% miss.	Complete	20% miss.	40% miss.
DSADS	MAESTRO	Original	0.86 ±.00	0.82 ±.01	0.78 ±.01	0.86 ±.00	0.82 ±.01	0.77 ±.01
	MAESTRO	Ours	0.87 ±.00	0.85 ±.01	0.79 ±.01	0.86 ±.00	0.83 ±.01	0.78 ±.01
	ShaSpec	Original	0.81 ±.01	0.45 ±.02	0.22 ±.01	0.79 ±.00	0.45 ±.02	0.20 ±.02
	ShaSpec	Ours	0.78 ±.00	0.75 ±.00	0.64 ±.03	0.77 ±.00	0.74 ±.00	0.64 ±.03
	FlexMoE	Original	0.67 ±.01	0.35 ±.09	0.19 ±.06	0.63 ±.03	0.34 ±.08	0.17 ±.07

Appendix

Table 8: Performance under complete and missing modalities. Accuracy and F1-score (

\uparrow

) on DSADS, DaliaHAR, WESAD, and PAMAP2 across three backbones (MAESTRO, ShaSpec, FlexMoE) under Complete , 20% missing , and 40% missing conditions under their recommended training regime and our proposed asymmetric training regime.

Modality	ShaSpec	FlexMoE	MAESTRO
wrist_ACC	$0.80\pm 0.00$	$0.70\pm 0.18$	$0.65\pm 0.05$
wrist_BVP	$0.47\pm 0.29$	$0.19\pm 0.01$	$0.45\pm 0.05$
wrist_EDA	$0.71\pm 0.16$	$0.22\pm 0.01$	$0.13\pm 0.01$
wrist_TEMP	$0.17\pm 0.01$	$0.22\pm 0.02$	$0.17\pm 0.02$
chest_ACC	$0.80\pm 0.00$	$0.58\pm 0.22$	$0.60\pm 0.03$

Appendix

Table 9: DaliaHAR – modality reliance scores

\rho_{j}

Modality	ShaSpec	FlexMoE	MAESTRO
torso	$0.42\pm 0.18$	$0.42\pm 0.03$	$0.40\pm 0.05$
right_arm	$0.37\pm 0.09$	$0.40\pm 0.01$	$0.39\pm 0.06$
left_arm	$0.45\pm 0.11$	$0.42\pm 0.01$	$0.41\pm 0.07$
right_leg	$0.39\pm 0.05$	$0.39\pm 0.02$	$0.37\pm 0.03$
left_leg	$0.37\pm 0.16$	$0.37\pm 0.03$	$0.45\pm 0.10$

Appendix

Table 10: DSADS — modality reliance weights

\rho_{j}

Modality	ShaSpec	FlexMoE	MAESTRO
chest_ACC	$0.80\pm 0.00$	$0.70\pm 0.14$	$0.78\pm 0.02$
chest_ECG	$0.80\pm 0.00$	$0.58\pm 0.08$	$0.00\pm 0.00$
chest_EMG	$0.01\pm 0.00$	$0.18\pm 0.00$	$0.00\pm 0.00$
chest_RESP	$0.70\pm 0.17$	$0.30\pm 0.01$	$0.00\pm 0.00$
chest_EDA	$0.73\pm 0.12$	$0.28\pm 0.02$	$0.49\pm 0.05$
chest_TEMP	$0.65\pm 0.14$	$0.23\pm 0.02$	$0.02\pm 0.03$

Appendix

Table 11: WESAD — modality reliance weights

\rho_{j}

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	torso	miss	0.803	0.799	0.788	0.796	0.779	0.792	0.786	0.805	0.741	0.794
		wave	0.763	0.772	0.782	0.757	0.761	0.787	0.759	0.767	0.734	0.799
		nonlin	0.850	0.854	0.851	0.847	0.849	0.853	0.849	0.856	0.822	0.840
		freq	0.850	0.854	0.862	0.862	0.843	0.865	0.855	0.859	0.819	0.856
		nonstat	0.809	0.783	0.790	0.793	0.783	0.787	0.783	0.789	0.745	0.804
	right_arm	miss	0.823	0.826	0.835	0.827	0.820	0.807	0.832	0.829	0.750	0.824

Appendix

Table 12: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DSADS with the MAESTRO backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.856.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	torso	miss	0.531	0.520	0.521	0.506	0.516	0.522	0.529	0.519	0.541	0.708
		wave	0.491	0.468	0.475	0.462	0.463	0.487	0.476	0.493	0.585	0.689
		nonlin	0.617	0.602	0.632	0.615	0.610	0.625	0.618	0.611	0.686	0.720
		freq	0.748	0.726	0.742	0.709	0.740	0.735	0.715	0.751	0.706	0.721
		nonstat	0.719	0.676	0.699	0.658	0.693	0.688	0.670	0.696	0.653	0.727
	right_arm	miss	0.582	0.587	0.567	0.463	0.560	0.555	0.543	0.580	0.639	0.680

Appendix

Table 13: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DSADS with the ShaSpec backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.765.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	torso	miss	0.680	0.691	0.695	0.671	0.680	0.701	0.679	0.682	0.644	0.719
		wave	0.670	0.697	0.706	0.696	0.700	0.704	0.698	0.710	0.613	0.719
		nonlin	0.693	0.693	0.682	0.679	0.683	0.698	0.684	0.687	0.637	0.709
		freq	0.703	0.733	0.733	0.737	0.725	0.738	0.715	0.739	0.660	0.718
		nonstat	0.670	0.713	0.705	0.690	0.685	0.709	0.694	0.703	0.626	0.685
	right_arm	miss	0.683	0.697	0.686	0.696	0.688	0.694	0.675	0.676	0.646	0.699

Appendix

Table 14: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DSADS with the Flex-MoE backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.740.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	wrist_ACC	miss	0.751	0.765	0.767	0.748	0.750	0.771	0.733	0.757	0.699	0.768
		wave	0.756	0.785	0.781	0.760	0.756	0.781	0.735	0.755	0.714	0.779
		nonlin	0.756	0.775	0.777	0.762	0.757	0.774	0.736	0.765	0.722	0.776
		freq	0.749	0.773	0.773	0.754	0.748	0.760	0.738	0.739	0.715	0.774
		nonstat	0.750	0.771	0.772	0.749	0.748	0.766	0.727	0.751	0.704	0.771
	wrist_BVP	miss	0.787	0.803	0.800	0.787	0.785	0.796	0.768	0.788	0.757	0.797

Appendix

Table 15: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DaliaHAR with the MAESTRO backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.788.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	wrist_ACC	miss	0.720	0.738	0.733	0.724	0.719	0.726	0.690	0.726	0.637	0.731
		wave	0.557	0.575	0.573	0.562	0.548	0.561	0.520	0.547	0.505	0.662
		nonlin	0.580	0.597	0.588	0.576	0.571	0.573	0.528	0.595	0.531	0.599
		freq	0.586	0.597	0.593	0.585	0.588	0.586	0.566	0.593	0.566	0.592
		nonstat	0.603	0.627	0.623	0.624	0.611	0.618	0.587	0.617	0.585	0.652
	wrist_BVP	miss	0.740	0.749	0.747	0.740	0.736	0.740	0.714	0.746	0.711	0.749

Appendix

Table 16: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DaliaHAR with the ShaSpec backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.741.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	wrist_ACC	miss	0.743	0.787	0.784	0.748	0.758	0.780	0.726	0.711	0.693	0.788
		wave	0.622	0.653	0.656	0.608	0.637	0.680	0.590	0.552	0.561	0.654
		nonlin	0.624	0.644	0.650	0.608	0.659	0.698	0.627	0.595	0.608	0.650
		freq	0.675	0.736	0.738	0.692	0.676	0.710	0.631	0.613	0.612	0.729
		nonstat	0.586	0.631	0.628	0.604	0.611	0.639	0.573	0.554	0.571	0.625
	wrist_BVP	miss	0.767	0.801	0.809	0.786	0.787	0.801	0.752	0.738	0.745	0.808

Appendix

Table 17: Full per-modality-subset, per-noise F1-score (

\uparrow

) on DaliaHAR with the Flex-MoE backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.768.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	IMU_ankle	miss	0.940	0.938	0.946	0.943	0.935	0.940	0.937	0.941	0.901	0.935
		wave	0.926	0.931	0.932	0.926	0.927	0.929	0.924	0.931	0.897	0.925
		nonlin	0.940	0.933	0.938	0.932	0.941	0.935	0.938	0.943	0.903	0.939
		freq	0.941	0.942	0.942	0.936	0.938	0.938	0.939	0.939	0.918	0.943
		nonstat	0.943	0.938	0.943	0.936	0.932	0.937	0.937	0.942	0.892	0.932
	IMU_hand	miss	0.944	0.941	0.945	0.933	0.937	0.944	0.939	0.945	0.910	0.946

Appendix

Table 18: Full per-modality-subset, per-noise F1-score (

\uparrow

) on PAMAP2 with the MAESTRO backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.951.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	IMU_ankle	miss	0.819	0.812	0.808	0.763	0.802	0.811	0.803	0.803	0.804	0.824
		wave	0.779	0.764	0.764	0.725	0.770	0.759	0.757	0.767	0.784	0.798
		nonlin	0.270	0.223	0.239	0.214	0.267	0.242	0.289	0.265	0.292	0.511
		freq	0.860	0.850	0.845	0.807	0.851	0.843	0.833	0.843	0.849	0.847
		nonstat	0.850	0.834	0.838	0.794	0.839	0.841	0.828	0.843	0.833	0.822
	IMU_hand	miss	0.839	0.829	0.823	0.816	0.833	0.826	0.825	0.830	0.837	0.845

Appendix

Table 19: Full per-modality-subset, per-noise F1-score (

\uparrow

) on PAMAP2 with the ShaSpec backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.862.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	IMU_ankle	miss	0.855	0.865	0.862	0.834	0.861	0.858	0.863	0.864	0.824	0.862
		wave	0.844	0.864	0.863	0.827	0.862	0.856	0.866	0.859	0.793	0.867
		nonlin	0.709	0.685	0.689	0.684	0.693	0.680	0.678	0.683	0.608	0.685
		freq	0.862	0.874	0.880	0.849	0.866	0.882	0.870	0.885	0.819	0.877
		nonstat	0.861	0.870	0.871	0.830	0.874	0.873	0.870	0.872	0.811	0.875
	IMU_hand	miss	0.883	0.887	0.890	0.864	0.888	0.885	0.888	0.891	0.870	0.892

Appendix

Table 20: Full per-modality-subset, per-noise F1-score (

\uparrow

) on PAMAP2 with the Flex-MoE backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.893.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	chest_EDA	miss	0.610	0.615	0.618	0.615	0.617	0.606	0.611	0.605	0.640	0.627
		wave	0.634	0.629	0.627	0.626	0.635	0.627	0.641	0.630	0.638	0.628
		nonlin	0.660	0.656	0.656	0.662	0.653	0.659	0.666	0.644	0.652	0.632
		freq	0.657	0.658	0.655	0.654	0.657	0.661	0.662	0.645	0.652	0.624
		nonstat	0.604	0.602	0.594	0.590	0.599	0.602	0.596	0.594	0.637	0.627
	chest_ACC	miss	0.658	0.656	0.656	0.655	0.655	0.653	0.663	0.646	0.665	0.637

Appendix

Table 21: Full per-modality-subset, per-noise F1-score (

\uparrow

) on WESAD with the MAESTRO backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.666.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
3-mod	C_ACC+ W_ACC+ W_EDA	miss	0.609	0.621	0.622	0.611	0.598	0.625	0.593	0.603	0.589	0.617
		wave	0.604	0.616	0.617	0.605	0.606	0.598	0.585	0.595	0.613	0.592
		nonlin	0.695	0.681	0.687	0.690	0.694	0.686	0.692	0.701	0.676	0.687
		freq	0.653	0.652	0.652	0.651	0.651	0.643	0.642	0.642	0.620	0.653
		nonstat	0.615	0.636	0.628	0.617	0.623	0.628	0.600	0.616	0.591	0.635
	C_ECG+ C_EMG+ C_RESP	miss	0.658	0.657	0.659	0.658	0.660	0.652	0.647	0.653	0.664	0.624

Appendix

Table 31

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	chest_EDA	miss	0.506	0.505	0.511	0.505	0.507	0.514	0.497	0.496	0.454	0.523
		wave	0.493	0.497	0.496	0.497	0.493	0.501	0.484	0.492	0.465	0.520
		nonlin	0.523	0.521	0.527	0.514	0.520	0.524	0.515	0.506	0.463	0.545
		freq	0.532	0.530	0.539	0.526	0.527	0.531	0.515	0.521	0.478	0.548
		nonstat	0.511	0.506	0.512	0.508	0.511	0.513	0.498	0.501	0.464	0.531
	chest_ACC	miss	0.547	0.547	0.550	0.555	0.542	0.547	0.542	0.551	0.493	0.570

Appendix

Table 22: Full per-modality-subset, per-noise F1-score (

\uparrow

) on WESAD with the ShaSpec backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.554.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
3-mod	C_ACC+ W_ACC+ W_EDA	miss	0.533	0.534	0.538	0.526	0.537	0.542	0.523	0.526	0.435	0.553
		wave	0.513	0.519	0.518	0.524	0.524	0.523	0.516	0.528	0.477	0.540
		nonlin	0.544	0.537	0.544	0.539	0.541	0.544	0.527	0.526	0.488	0.563
		freq	0.537	0.538	0.545	0.531	0.534	0.541	0.517	0.522	0.472	0.546
		nonstat	0.505	0.511	0.510	0.514	0.505	0.509	0.494	0.508	0.445	0.530
	C_ECG+ C_EMG+ C_RESP	miss	0.555	0.547	0.560	0.544	0.553	0.557	0.551	0.536	0.486	0.570

Appendix

Table 33

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
1-mod	chest_EDA	miss	0.519	0.536	0.535	0.515	0.534	0.556	0.516	0.509	0.492	0.520
		wave	0.515	0.534	0.540	0.511	0.530	0.549	0.501	0.512	0.506	0.522
		nonlin	0.505	0.530	0.524	0.544	0.504	0.517	0.500	0.489	0.484	0.519
		freq	0.532	0.539	0.554	0.561	0.525	0.553	0.492	0.502	0.505	0.543
		nonstat	0.515	0.538	0.538	0.526	0.523	0.538	0.482	0.509	0.495	0.524
	chest_ACC	miss	0.526	0.543	0.555	0.550	0.501	0.499	0.506	0.525	0.540	0.550

Appendix

Table 23: Full per-modality-subset, per-noise F1-score (

\uparrow

) on WESAD with the Flex-MoE backbone. Each cell is the F1 for one specific (corrupted modality subset, noise type) combination. Noise abbreviations: miss =discrete missingness, wave =waveform anomaly, nonlin =nonlinear degradation, freq =frequency attenuation, nonstat =nonstationary noise. Clean baseline: F1 = 0.554.

Card.	Subset	Noise	Static	TENT	EATA	READ	SuMi	SoTTA	AeO	PTA	PSDNorm	Ours
3-mod	C_ACC+ W_ACC+ W_EDA	miss	0.510	0.502	0.493	0.503	0.489	0.474	0.493	0.516	0.455	0.501
		wave	0.522	0.516	0.522	0.508	0.516	0.504	0.530	0.520	0.511	0.522
		nonlin	0.539	0.564	0.553	0.518	0.490	0.501	0.465	0.489	0.504	0.552
		freq	0.526	0.530	0.527	0.549	0.547	0.568	0.537	0.516	0.510	0.526
		nonstat	0.464	0.473	0.477	0.455	0.448	0.448	0.441	0.427	0.445	0.476
	C_ECG+ C_EMG+ C_RESP	miss	0.548	0.574	0.589	0.555	0.523	0.544	0.480	0.508	0.527	0.571

Appendix

Table 35

Backbone	Method	Latency (ms)	Energy (J)	GFLOPs	Trainable Params
MAESTRO
	Static	435.0	381.1	5.56	0
	TENT	1200.2	1254.5	53.63	3584
	EATA	1038.6	1108.4	53.63	3584
	READ	554.8	528.6	12.59	49920
	SuMi	1238.4	1319.8	53.36	3584

Appendix

Table 24: Efficiency metrics on DSADS.

Dataset	Method	Latency (ms)	Energy (J)	GFLOPs	Trainable Params
DaliaHAR / MAESTRO
	Static	141.2	147.8	1.44	0
	TENT	508.3	597.9	13.84	1536
	EATA	480.7	554.4	13.84	1536
	READ	177.4	184.8	3.20	6336
	SuMi	521.4	560.2	12.22	1536

Appendix

Table 25: Efficiency metrics on DaliaHAR and PAMAP2 with MAESTRO.

		Offline training		Detector inference
Modality	Parameters	Time (s)	GFLOPs	Latency (ms)	GFLOPs
Wrist acceleration	8,512	35.3	97.7	33.9	0.0032
Wrist BVP	8,384	42.2	32.6	12.2	0.0011
Wrist EDA	8,384	42.1	32.6	12.2	0.0011
Wrist temperature	8,384	42.3	32.6	12.1	0.0011
Chest acceleration	8,512	42.6	97.7	33.9	0.0032

Appendix

Table 26: Detector footprint on DaliaHAR–MAESTRO. Offline training cost and detector-only inference cost on an NVIDIA L40. Inference measurements are per batch of 64.

Implementation	Detector	End-to-end
Reference	467	1,713
Optimized	16	1,262

Appendix

Table 27: DSADS–MAESTRO detector optimization. Latency in milliseconds per batch from a controlled rerun. End-to-end latency includes detection, adaptation, and inference.

Figure 12: Gate lr sensitivity on DaliaHAR,

\mathcal{S}_{i}=\{\text{wrist\_ACC},\text{wrist\_Temp}\}

. (a)

\eta_{\alpha}=10^{-3}

(F1

=

0.65): gates pinned at

\sim 0.5

. (b)

\eta_{\alpha}=5\times 10^{-2}

(F1

=

0.67): gates move within the episodic budget.

Dataset	Static	Unconditional drop	CARAT
DSADS	0.69 $\pm$ 0.22	0.67 $\pm$ 0.22	0.72 $\pm$ 0.20
DaliaHAR	0.74 $\pm$ 0.07	0.67 $\pm$ 0.11	0.76 $\pm$ 0.07
PAMAP2	0.92 $\pm$ 0.04	0.68 $\pm$ 0.36	0.92 $\pm$ 0.04
WESAD	0.65 $\pm$ 0.04	0.62 $\pm$ 0.03	0.63 $\pm$ 0.04

Appendix

Table 28: Unconditional drop vs. CARAT on MAESTRO. F1-score (

\uparrow

) averaged across 10 corrupted-modality scenarios

\times

5 noise types per dataset. Mean

{}_{\pm\text{std}}

across all (subset, noise) combinations. Bold : higher of {Unconditional drop, CARAT } per row. We restrict this analysis to MAESTRO, the most stable missingness-resilient backbone in our benchmark, where the routing-vs-dropping mechanism is evaluated without confounding from backbone-specific fragility under modality dropout.

Setting	$n$	Comparator	Reported $\Delta$	Reported 95% CI	Reported $p$	W/T/L
DSADS / MAESTRO	50	PTA	$+0.20$	$[-0.60,\,+1.90]$	$1.00$	5/4/0
DSADS / ShaSpec	50	PSDNorm	$+7.00$	$[+5.30,\,+9.10]$	$6\!\times\!10^{-8}$	9/0/0
DSADS / Flex-MoE	50	PTA	$+0.00$	$[-0.40,\,+0.70]$	$1.00$	2/7/0
DaliaHAR / MAESTRO	50	EATA	$-0.20$	$[-0.40,\,-0.10]$	$0.008$	7/0/2
DaliaHAR / ShaSpec	50	TENT	$+1.80$	$[+0.90,\,+3.70]$	$6\!\times\!10^{-4}$	9/0/0
DaliaHAR / Flex-MoE	50	EATA	$-0.20$	$[-0.40,\,+0.00]$	$0.19$	7/2/0

Appendix

Table 29: Paired condition-level comparisons.

n

is the number of matched corruption conditions. The reported paired effect and 95% interval compare CARAT with the named setting-specific comparator. Estimates, intervals, and

p

-values are retained from the original paired analysis. W/T/L counts the reported test outcomes across nine alternatives, not wins across the

n

conditions. Effect sizes and intervals are in macro-F1 points; positive values favor CARAT .

Comparator	Mean F1	W/T/L	Median $\Delta$	Reported 95% CI	Reported $p$
Static	$69.30$	36/0/14	$+2.70$	$[+1.30,\,+4.80]$	$6\!\times\!10^{-5}$
TENT	$69.30$	36/1/13	$+2.40$	$[+1.20,\,+4.40]$	$9\!\times\!10^{-5}$
EATA	$69.30$	34/4/12	$+2.40$	$[+1.10,\,+4.50]$	$9\!\times\!10^{-5}$
READ	$69.00$	38/0/12	$+3.10$	$[+1.60,\,+5.60]$	$5\!\times\!10^{-5}$
SuMi	$69.10$	37/1/12	$+3.00$	$[+1.60,\,+5.30]$	$6\!\times\!10^{-5}$
SoTTA	$69.10$	35/1/14	$+2.70$	$[+1.30,\,+4.90]$	$7\!\times\!10^{-5}$

Appendix

Table 30: Experiment R3.1: DSADS–MAESTRO under five corruption redraws. Mean F1, paired differences and interval bounds use the 0–100 scale. CARAT achieves 73.80 mean F1. W/T/L sums to 50 matched conditions; the estimates, intervals and

p

-values retain the original repeated-evaluation reporting.

Corruption	Rank $\downarrow$
Waveform anomaly	3.19
Discrete missingness	3.25
Nonstationary noise	3.49
Nonlinear degradation	4.03
Frequency attenuation	4.62

Appendix

Table 31: Condition-level rank breakdowns. Left: CARAT across the five corruption families. Right: CARAT and the lowest-rank alternative within each backbone. The reported means pool the conditions in the indicated group.

Method	DSADS	DaliaHAR	PAMAP2	WESAD	Pooled	Worst
CARAT	3.56	2.39	4.09	4.32	3.72	4.32
EATA	5.30	2.68	4.47	3.70	3.97	5.30
TENT	5.53	2.46	4.65	4.25	4.22	5.53
SoTTA	5.00	4.57	5.41	4.49	4.79	5.41
Static	5.51	5.96	4.60	4.89	5.19	5.96
SuMi	6.02	6.92	5.56	5.71	6.00	6.92

Appendix

Table 32: Mean condition-level ranks by dataset. Smaller is better. Pooled averages all 720 condition ranks; Worst is the largest of the four dataset-level ranks. These are not the ranks of dataset–backbone means in Table 1 . Bold and underline mark the lowest and second-lowest displayed ranks per column.

Setting	Baseline range	CARAT
DSADS / ShaSpec	$18.80$ – $21.60$	$12.30$
PAMAP2 / ShaSpec	$20.20$ – $21.90$	$16.20$
DSADS / MAESTRO	$12.80$ – $13.50$	$10.20$
DSADS / Flex-MoE	$9.90$ – $10.40$	$9.20$

Appendix

Table 33: Within-subset dispersion across corruption families. Each entry averages the sample standard deviation over the five corruption scores within a fixed modality-subset scenario. Values are in macro-F1 points. The baseline column gives the reported range across the seven parameter-updating baselines.

Source training	0% missing	20% missing	40% missing
Uniform (Original)	0.77 ±0.02	0.57 ±0.02	0.39 ±0.01
OGM	0.73 ±0.00	0.50 ±0.01	0.35 ±0.02
OPM	0.76 ±0.00	0.57 ±0.02	0.45 ±0.01
MIAM	–	0.68	0.59
Asymmetric (ours)	0.74 ±0.00	0.68 ±0.02	0.61 ±0.02

Appendix

Table 34: Source-training replacement on DaliaHAR–ShaSpec. Macro-F1 (

\uparrow

), with the reported dispersions where available. Uniform/asymmetric entries use the original three-seed source-training evaluation. MIAM reports means only at 20%/40% missingness; – denotes an unreported result. Bold / underline : best/second distinct displayed mean per column.

	OGM source training			OPM source training			MIAM source training
Method	w_acc	all_w	Div.	w_acc	all_w	Div.	w_acc	all_w	Div.
Static	0.55 ±0.10	0.53 ±0.13	0.57 ±0.08	0.58 ±0.10	0.55 ±0.15	0.59 ±0.09	0.60 ±0.08	0.55 ±0.13	0.58 ±0.08
TENT	0.60 ±0.15	0.57 ±0.18	0.62 ±0.10	0.63 ±0.16	0.57 ±0.21	0.63 ±0.12	0.64 ±0.15	0.57 ±0.21	0.61 ±0.11
EATA	0.61 ±0.14	0.57 ±0.18	0.62 ±0.10	0.63 ±0.13	0.58 ±0.19	0.63 ±0.10	0.64 ±0.11	0.57 ±0.17	0.61 ±0.09
READ	0.60 ±0.16	0.58 ±0.17	0.62 ±0.10	0.60 ±0.18	0.57 ±0.18	0.62 ±0.11	0.63 ±0.12	0.55 ±0.19	0.60 ±0.10
SuMi	0.56 ±0.11	0.54 ±0.14	0.58 ±0.08	0.58 ±0.13	0.54 ±0.16	0.59 ±0.10	0.59 ±0.11	0.53 ±0.16	0.57 ±0.09

Appendix

Table 35: Complete TTA comparison on replacement checkpoints. DaliaHAR–ShaSpec macro-F1 (

\uparrow

), reported mean ±SD across the evaluated corruption conditions. all_w : all_wrist ; Div.: pooled diverse setting. The spreads describe variation across conditions, not confidence intervals over independent training runs. Styling is applied within each column to displayed means; rounded ties are not statistical equivalence.

	Terminal dropout profile $\rho^{j}$
White-noise modalities	w_acc	w_bvp	w_eda	w_temp	c_acc	F1 points
None	0.75	0.80	0.66	0.40	0.80	80.20 ±1.40
w_temp	0.80	0.80	0.44	0.01	0.80	81.40
w_temp, w_eda	0.80	0.80	0.00	0.00	0.80	82.50
w_temp, w_eda, w_bvp	0.80	0.00	0.00	0.00	0.80	80.40

Appendix

Table 36: Variance-matched white-noise control on DaliaHAR–MAESTRO. Bold profile entries identify the replaced streams, not the best values.

\rho^{j}

is the terminal dropout probability, rather than a profile normalized to sum to one. F1 is displayed on a 0–100 scale. Only the unmodified control includes a reported three-seed spread.

	Normalized reliance $100\,r_{e^{\star}}^{j}$ (%)
Input projection	w_acc	w_bvp	w_eda	w_temp	c_acc	F1 points
Original widths	29.50 ±4.40	17.80 ±1.30	19.30 ±3.10	9.80 ±4.10	23.70 ±13.00	80.30 ±0.70
c_acc $\times 8$	22.20 ±2.90	17.80 ±1.70	12.10 ±9.00	21.30 ±6.20	26.60 ±11.60	80.20 ±0.70
w_temp $\times 8$	19.30 ±4.30	28.20 ±7.80	14.60 ±5.70	14.60 ±11.50	23.40 ±3.80	79.50 ±2.10

Appendix

Table 37: Input-projection widening on DaliaHAR–MAESTRO. Reported mean ±SD ; bold identifies the widened modality. Reliance ratios are percentages; F1 uses a 0–100 scale. These are separate retraining controls from the white-noise experiment.

Explore similar work

May 6, 2026cs.CV

Temporal Structure Matters for Efficient Test-Time Adaptation in Wearable Human Activity Recognition

Wearable human activity recognition (WHAR) models often suffer from performance degradation under real-world cross-user distribution shifts. Test-time adaptation (TTA) mitigates this degradation by adapting models online using unlabeled test streams, yet existing methods largely inherit assumptions from vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams. In this paper, we revisit such temporal structure as a feature-conditioned inference signal rather than merely an output-space smoothing prior. We derive the insight that temporal continuity and observation-induced feature deviations provide complementary cues for determining when to preserve or release temporal inertia and where to route prediction refinement during likely transitions. Building upon this insight, we propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment. SIGHT estimates predictive surprise by comparing the current feature with a prototype-based expected state, and then uses the resulting feature deviation to guide geometry-aware transition routing based on prototype alignment and stream-level marginal habit tracking. Evaluations on real-world datasets confirm that SIGHT outperforms existing TTA baselines while reducing computational and memory costs.

Zishu Zhou, Zaipeng Xie, Xuanyao Jie

College of Computer Science and Software Engineering, Hohai University

Human Activity Recognition Test-Time Adaptation

Ask PDF

Apr 21, 2026cs.CV

Multi-modal Test-time Adaptation via Adaptive Probabilistic Gaussian Calibration

Multi-modal test-time adaptation (TTA) enhances the resilience of benchmark multi-modal models against distribution shifts by leveraging the unlabeled target data during inference. Despite the documented success, the advancement of multi-modal TTA methodologies has been impeded by a persistent limitation, i.e., the lack of explicit modeling of category-conditional distributions, which is crucial for yielding accurate predictions and reliable decision boundaries. Canonical Gaussian discriminant analysis (GDA) provides a vanilla modeling of category-conditional distributions and achieves moderate advancement in uni-modal contexts. However, in multi-modal TTA scenario, the inherent modality distribution asymmetry undermines the effectiveness of modeling the category-conditional distribution via the canonical GDA. To this end, we introduce a tailored probabilistic Gaussian model for multi-modal TTA to explicitly model the category-conditional distributions, and further propose an adaptive contrastive asymmetry rectification technique to counteract the adverse effects arising from modality asymmetry, thereby deriving calibrated predictions and reliable decision boundaries. Extensive experiments across diverse benchmarks demonstrate that our method achieves state-of-the-art performance under a wide range of distribution shifts. The code is available at https://github.com/XuJinglinn/AdaPGC.

Jinglin Xu, Yi Li, Chuxiong Sun +3

Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences · National Defense University

Modalities Conditional Distribution

Ask PDF

May 13, 2026cs.LG

VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals

Wearable devices enable continuous health monitoring from multimodal signals, but real-world deployment is hindered by limited labeled data and pervasive sensor incompleteness. While large-scale self-supervised pretraining reduces label dependence, most existing methods assume full modality availability. Current approaches for handling modality missingness often reconstruct entire absent signals, which can encourage hallucinating modality-specific details that are not inferable from the observed sensor signals and degrade robustness. We propose VCR, a self-supervised framework that learns to extract valid representations robust to modality missingness. VCR employs an orthogonal tokenizer to enforce strict orthogonal disentanglement by rectifying latent manifolds and applying a geometric projection, separating each modality into shared semantics and modality-specific residuals. This design preserves complete information integrity while serving as a structural foundation for robust learning under modality missingness. The resulting tokens are processed by a missing-aware mixture-of-experts backbone that adapts to varying patterns of modality availability. By constraining the objective to reconstruct only the shared components of missing modalities, VCR effectively mitigates hallucinations of non-inferable modality-specific details. Across multiple health monitoring tasks, VCR consistently improves performance and robustness under full, single-missing, and multiple-missing modality settings compared with strong supervised and self-supervised baselines.

Yuxuan Weng, Wenhan Luo, Qijia Shao

The Hong Kong University of Science and Technology (HKUST)

Missing Modalities Wearable Physiology Multimodal Learning Self-Supervised Learning

Ask PDF