Information Maximization for Long-Tailed Semi-Supervised Domain Generalization

Leo Fillioux; Omprakash Chakraborty; Quentin Gopée; Pierre Marza; Paul-Henry Cournède; Stergios Christodoulidis; Maria Vakalopoulou; Ismail Ben Ayed

cs.CVMar 9, 2026

Information Maximization for Long-Tailed Semi-Supervised Domain Generalization

Authors: Leo Fillioux, Omprakash Chakraborty, Quentin Gopée, Pierre Marza, Paul-Henry Cournède, Stergios Christodoulidis, Maria Vakalopoulou, Ismail Ben Ayed,

Organizations: Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, CDSU, IHU PRISM, Gif-sur-Yvette · LIVIA, ILLS, ETS Montréal

Abstract

Semi-supervised domain generalization (SSDG) has recently emerged as an appealing alternative to tackle domain generalization when labeled data is scarce but unlabeled samples across domains are abundant. In this work, we identify an important limitation that hampers the deployment of state-of-the-art methods on more challenging but practical scenarios. In particular, state-of-the-art SSDG severely suffers in the presence of long-tailed class distributions, an arguably common situation in real-world settings. To alleviate this limitation, we propose IMaX, a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, where the Mutual Information (MI) between the learned features and latent labels is maximized, constrained by the supervision from the labeled samples. Our formulation integrates an α-entropic objective, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions. IMaX can be seamlessly plugged into recent state-of-the-art SSDG, consistently enhancing their performance, as demonstrated empirically across two different image modalities.

Figures & tables

Panel of Figure 1 — Figure 1: Performance degradation under class imbalance. Impact on the accuracy of having long-tailed class distribution in the training data using the FBCSA [ 9 ] trainer ( left ). Impact of the imbalance factor $\gamma$ (see Section 4.1 ) on the performance of FBCSA and our proposed approach ( right ).

		ESCA		Retina
	Method	$m_{L}=5$	$m_{L}=10$	$m_{L}=5$	$m_{L}=10$
Baseline	FixMatch/ + IMaX	$61.0\>/\>\bf 68.3$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+7.3}}}}$	$69.2\>/\>\bf 73.1$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.9}}}}$	$31.4\>/\>\bf 38.4$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+7.0}}}}$	$34.4\>/\>\bf 40.2$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+5.8}}}}$
	FreeMatch/ + IMaX	$59.3\>/\>\bf 63.2$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.9}}}}$	$67.5\>/\>\bf 71.0$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.5}}}}$	$36.0\>/\>\bf 38.7$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+2.7}}}}$	$34.0\>/\>\bf 37.0$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.0}}}}$
	StyleMatch/ + IMaX	$60.4\>/\>\bf 67.7$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+7.3}}}}$	$69.0\>/\>\bf 73.4$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+4.4}}}}$	$30.9\>/\>\bf 36.0$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+5.1}}}}$	$34.6\>/\>\bf 38.4$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.8}}}}$
FBCSA	FixMatch/ + IMaX	$64.0\>/\>\bf 67.7$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.7}}}}$	$72.2\>/\>\bf 73.2$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+1.0}}}}$	$28.8\>/\>\bf 32.3$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.5}}}}$	$\textbf{32.0}\>/\>29.5$ ${}_{{\textbf{{\color[rgb]{0.55,0.22,0.22}-2.5}}}}$
	FreeMatch/ + IMaX	$63.1\>/\>\bf 65.5$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+2.4}}}}$	$71.6\>/\>\bf 72.4$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+0.8}}}}$	$30.1\>/\>\bf 31.1$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+1.0}}}}$	$27.9\>/\>\bf 29.8$ ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+1.9}}}}$

Table 1: Performance in the long-tailed SSDG scenario. We report [standard / + IMaX ], where + IMaX denotes the results obtained by our approach when applied on each SSDG method and across the three SSL strategies.

Loss	$m_{L}=5$	$m_{L}=10$
SSL baseline	61.0	69.2
SSL baseline + $(-\mathcal{H}(Y))$ (Eq. 6 )	66.0 ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+5.0}}}}$	72.6 ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.4}}}}$
SSL baseline + $(-\mathcal{H}_{\alpha}(Y))$ (Eq. 8 ) = IMaX	68.3 ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+7.3}}}}$	73.1 ${}_{{\textbf{{\color[rgb]{0.18,0.407,0.266}+3.9}}}}$

Table 2: Ablation on the proposed learning objectives, where SSL baseline can be defined as

\mathcal{H}(Y|X_{L})+\mathcal{H}(\hat{Y}|X_{U})

. Comparison on the ESCA dataset.

Panel of Figure 2 — Figure 2: Ablation on the $\boldsymbol{\alpha}$ parameter. Impact on the $\alpha$ parameter of $\mathcal{H}_{\alpha}$ on the accuracy for the ESCA ( left ) Retina ( right ) datasets under long-tailed distribution. Results are for FreeMatch on the SSL Baseline setting.

Domain Generalization

Domain generalization develops models that perform reliably in new environments without training on target-domain data.

See all 375 papers

When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts
Jul 19, 2025Wooseok Ha, Yuansi ChenFine-Tuning Domain Adaptation
Department of Mathematical Sciences, KAIST · Department of Mathematics, ETH Zürich
Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning
Aug 31, 2026Yue Cheng, Jiajun Zhang, Xiaohui Gao +2Class-Imbalanced Learning Semi-Supervised Learning
Beijing Jiaotong University · AntGroup · Northwestern Polytechnical University +1
Towards Multimodal Domain Generalization with Few Labels
Feb 26, 2026Hongzhao Li, Hao Dong, Hualei Wan +3Multimodal Domain Generalization Multimodal Learning
Zhengzhou University · ETH Zürich · MBZUAI

Jul 19, 2025J/K move · Enter open · S save

When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts

Wooseok Ha, Yuansi Chen

Department of Mathematical Sciences, KAIST · Department of Mathematics, ETH Zürich

Fine-Tuning Domain Adaptation

Open paper AskPDF arXiv

Semi-supervised domain adaptation (SSDA) seeks to achieve accurate predictions in a target domain with limited labeled target data by exploiting abundant source and unlabeled target data. We study this problem under structural causal models (SCMs), which provide a statistical framework to describe distribution shifts between source and target domains as interventions in the data-generating process rather than ad hoc changes in model parameters. The central phenomenon is that, under low-dimensional interventions, source and unlabeled target data can help identify the high-dimensional shared structure, leaving only a low-dimensional target-specific correction to be learned from limited labeled target data. We formalize this principle for three canonical intervention models and propose the corresponding SSDA methods FT-DIP, FT-OLS-Src and FT-CIP. Under each intervention model, we demonstrate how extending an unsupervised domain adaptation (UDA) method to SSDA can achieve minimax-optimal target performance with limited target labels, with the labeled-target sample complexity scaling with the intervention dimension rather than the ambient dimension. When the distribution shift is underspecified, we propose the Multi-Adaptive-Start Fine-Tuning (MASFT) algorithm, which fine-tunes from multiple adaptive starts and selects among them using a small target validation set, incurring only logarithmic overhead in the number of starts. We validate the effectiveness of our proposed methods through simulated and real data experiments.

Abstract

Domain Generalization

Explore similar work

When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts

Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning

Towards Multimodal Domain Generalization with Few Labels