Generative Classifier for Domain Generalization

Shaocong Long; Qianyu Zhou; Dexuan Zhang; Tatsuya Harada

cs.CVApr 3, 2025

Generative Classifier for Domain Generalization

Authors: Shaocong Long, Qianyu Zhou, Dexuan Zhang, Tatsuya Harada

Organizations: Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China · College of Computer Science and Technology and the Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun, China

Abstract

Domain generalization (DG) aims to improve the generalizability of computer vision models toward distribution shifts. The mainstream DG methods predominantly focus on learning domain invariance across domains, however, such methods overlook the untapped potential inherent in domain-specific information. While the prevailing practice of discriminative linear classifier has been tailored to domain-invariant features, it struggles when confronted with diverse domain-specific information, \emph{e.g.,} intra-class shifts, that exhibits multi-modality. To address these issues, we explore the theoretical implications of relying on domain-invariant features, revealing the crucial role of domain-specific information in mitigating the target risk for DG. Drawing from these insights, we propose Generative Classifier-driven Domain Generalization (GCDG), introducing a generative paradigm for the DG classifier based on Gaussian Mixture Models (GMMs) for each class across domains. GCDG consists of three key modules: Heterogeneity Learning Classifier~(HLC), Spurious Correlation Blocking~(SCB), and Diverse Component Balancing~(DCB). Concretely, HLC attempts to model the feature distributions and thereby capture valuable domain-specific information via GMMs. SCB identifies the neural units containing spurious correlations and perturbs them, mitigating the risk of HLC learning irrelevant spurious patterns. Meanwhile, DCB ensures a balanced contribution of components within HLC, preventing the underestimation or neglect of critical components. In this way, GCDG excels in capturing the nuances of domain-specific information characterized by diverse distributions. Extensive experiments show GCDG's comparable performance on five DG benchmarks and one face anti-spoofing dataset, seamlessly integrating into existing DG methods with consistent improvements.

Figures & tables

Fig. 1: Comparison of modeling a class between the discriminative linear classifier and the proposed generative classifier in DG. (a) The prevailing linear classifier in DG operates under the assumption of unimodal distribution, encountering substantial challenges when confronted with domain-specific data that exhibits multi-modality. (b) In this paper, we introduce a novel generative classifier to capture the underlying multi-modal distribution present in domain-specific data.

Fig. 2: The framework of our proposed GCDG. The key innovation is the Heterogeneity Learning Classifier, which is a generative classifier consisting of a mixture of Gaussians for each class and adept at effectively harnessing valuable domain-specific information exhibiting multi-modality. Besides, we introduce Spurious Correlation Blocking to shuffle the neural units containing spurious correlations, mitigating their adverse effect on capturing domain-specific information. Furthermore, Diverse Component Balancing is designed to balance the contributions of diverse components, avoiding underestimating essential ones.

Methods	OfficeHome ( $\uparrow$ )	DomainNet ( $\uparrow$ )
ERM	60.51	43.68
GMMSeg	60.16	13.16
GCDG (ours)	64.49	46.60

TABLE I: Comparison of the generalizability on datasets where the number of samples in one class is small.

Model	PACS ( $\uparrow$ )	VLCS ( $\uparrow$ )	OH ( $\uparrow$ )	TI ( $\uparrow$ )	DN ( $\uparrow$ )	Avg. ( $\uparrow$ )
SAM [ 84 ]	96.64	85.01	79.30	91.25	64.44	83.33
SWAD [ 85 ]	96.20	84.44	78.53	90.90	64.44	82.90
PCL [ 22 ]	96.17	84.16	79.60	87.89	64.25	82.41
GCDG (ours)	96.98	85.52	79.62	92.75	64.91	83.96

TABLE II: Comparisons of in-domain generalization on five DG benchmarks with flatness-aware optimization methods.

Methods	Office-Home (Clipart)		PACS (Cartoon)
Methods	Entropy	Accuracy	Entropy	Accuracy
ERM	7.04	48.00	7.96	74.79
GCDG	7.62	51.27	8.66	78.58

TABLE III: Comparison of the average entropy values of features on source domains when the model is converged.

Parameter	Value
Learning rate	[5e-5, 8e-5]
Number of components $K$	[2, 3, 5]
Compression dimension $D$	[64, 1024]
Quantile $q$	[10, 20, 30]

TABLE IV: Hyperparameter search space.

Method	Target domain				Avg.( $\uparrow$ )
Method	Art	Cartoon	Photo	Sketch	Avg.( $\uparrow$ )
ResNet-18
RSC [ 94 ] (ECCV’2020)	79.88	76.87	94.56	77.11	82.10
MTL [ 95 ] (JMLR’2021)	$79.99$	$72.18$	$95.28$	$74.94$	$80.60$
SagNet [ 96 ] (CVPR’2021)	81.15	$75.05$	$94.61$	75.38	$81.55$
ARM [ 97 ] (NeurIPS’2021)	$80.42$	$75.96$	$95.21$	$72.33$	$80.98$

TABLE V: Generalization results of state-of-the-art methods and our GCDG on PACS.

Method	Target domain				Avg.( $\uparrow$ )
Method	L100	L38	L43	L46	Avg.( $\uparrow$ )
ResNet-18
RSC [ 94 ] (ECCV’2020)	$47.32$	$37.66$	$51.67$	35.95	$43.15$
MTL [ 95 ] (JMLR’2021)	$38.94$	$35.18$	$52.80$	$35.29$	$40.55$
SagNet [ 96 ] (CVPR’2021)	47.25	29.67	52.87	25.22	$38.75$
ARM [ 97 ] (NeurIPS’2021)	$44.98$	$33.73$	$43.39$	$27.77$	$37.47$

TABLE VI: Performance comparison with state-of-the-art methods on Terra-Incognita.

Method	Target domain				Avg.( $\uparrow$ )
Method	Art	Clipart	Product	Real	Avg.( $\uparrow$ )
ResNet-18
RSC [ 94 ] (ECCV’2020)	$49.38$	$45.91$	$66.84$	$67.41$	$57.38$
MTL [ 95 ] (JMLR’2021)	$52.58$	$46.99$	$70.83$	$72.46$	$60.72$
SagNet [ 96 ] (CVPR’2021)	56.28	51.32	$70.64$	73.38	$62.90$
ARM [ 97 ] (NeurIPS’2021)	$52.68$	$45.82$	$68.64$	$71.40$	$59.63$

TABLE VII: Performance comparison with state-of-the-art approaches on Office-Home.

Method	Target domain				Avg.( $\uparrow$ )
Method	Caltech	LabelMe	SUN	PASCAL	Avg.( $\uparrow$ )
ResNet-18
RSC [ 94 ] (ECCV’2020)	$93.29$	$64.47$	$71.52$	$73.31$	$75.65$
MTL [ 95 ] (JMLR’2021)	$96.38$	$62.54$	$70.91$	$71.68$	$75.38$
SagNet [ 96 ] (CVPR’2021)	97.09	$62.07$	$70.37$	75.42	76.24
ARM [ 97 ] (NeurIPS’2021)	$96.29$	$61.55$	$72.32$	$76.27$	$76.61$

TABLE VIII: Generalization results of state-of-the-art methods and our GCDG on VLCS.

Methods	I&C&M to O		O&C&I to M		O&M&I to C		Avg.
Methods	HTER(%)	AUC(%)	HTER(%)	AUC(%)	HTER(%)	AUC(%)	HTER(%)	AUC(%)
MADDG [ 63 ] (CVPR’2019)	27.98	80.02	17.69	88.06	24.50	84.51	23.39	84.20
D 2 AM [ 103 ] (AAAI’2021)	15.27	90.87	12.70	95.66	20.98	85.58	16.32	90.70
SSDG [ 104 ] (CVPR’2020)	25.17	81.83	16.67	90.47	23.11	85.45	21.65	85.92
RFM [ 105 ] (AAAI’2020)	16.45	91.16	13.89	93.98	20.27	88.16	16.87	91.1
DRDG [ 106 ] (IJCAI’2021)	15.63	91.75	12.43	95.81	19.05	88.79	15.70	92.12

TABLE IX: Comparison to the state-of-the-art FAS methods. The bold numbers indicate the best performance.

Method	Target domain						Avg.( $\uparrow$ )
Method	Clipart	Infograph	painting	Quickdraw	Real	Sketch
ResNet-50
RSC [ 94 ] (ECCV’2022)	55.0	18.3	44.4	12.2	55.7	47.8		38.9
MTL [ 95 ] (JMLR’2021)	57.9	18.5	46.0	12.5	59.5	49.2	40.6
SagNet [ 96 ] (CVPR’2021)	57.7	19.0	45.3	12.7	58.1	48.8	40.3
ARM [ 97 ] (NeurIPS’2021)	49.7	16.3	40.9	9.4	53.4	43.5	35.5

TABLE X: Performance comparison with state-of-the-art methods on DomainNet. † denotes reproduced results.

Model	PACS ( $\uparrow$ )	VLCS ( $\uparrow$ )	OH ( $\uparrow$ )	TI ( $\uparrow$ )	DN ( $\uparrow$ )	Avg. ( $\uparrow$ )
ERM	80.72	74.50	60.51	41.44	43.68	60.17
MLP-ERM	81.38	73.85	61.24	43.17	42.68	60.47
GCDG (ours)	83.44	76.86	64.49	44.86	46.63	63.26

TABLE XI: Generalization performance with various classifiers on five commonly used DG datasets. MLP-ERM denotes ERM with MLP-based classifier.

Fig. 4: Visualization of the loss landscapes for ERM, the flatness-aware method SWAD [ 85 ] , and the proposed GCDG on PACS. Note that the loss landscape is visualized on the source domains. Notably, our proposed GCDG exhibits superior efficacy in fostering flat minima compared to ERM and the flatness-aware method SWAD.

ID	HLC	DCB	SCB	Target domain				Avg.( $\uparrow$ )
ID	HLC	DCB	SCB	Art	Cartoon	Photo	Sketch	Avg.( $\uparrow$ )
A	-	-	-	78.76	74.79	96.29	73.02	80.72
B	✓	-	-	81.84	74.49	94.13	77.35	81.95
C	✓	✓	-	81.74	74.27	93.71	79.82	82.39
D	✓	-	✓	82.13	75.85	94.91	76.28	82.29
E	✓	✓	✓	83.06	78.50	92.63	79.56	83.44

TABLE XII: Ablation study of the proposed components in GCDG on PACS.

Models	Target domain				Avg.( $\uparrow$ )
Models	Art	Cartoon	Photo	Sketch	Avg.( $\uparrow$ )
DMG [ 35 ]	76.90	80.38	93.35	75.21	81.46
DDG [ 37 ]	79.30	74.00	91.80	75.80	80.20
DRM [ 68 ]	81.20	71.20	93.70	78.60	81.20
GCDG (ours)	83.06	78.50	92.63	79.56	83.44

TABLE XIII: Comparison between methods leveraging domain-specific information.

Models	Target domain				Avg.( $\uparrow$ )
Models	Art	Cartoon	Photo	Sketch	Avg.( $\uparrow$ )
ERM	78.76	74.79	96.29	73.02	80.72
+ GCDG	83.06	78.50	92.63	79.56	83.44
SWAD [ 85 ]	83.28	74.63	96.56	77.96	83.11
+ GCDG	85.17	78.04	95.14	78.28	84.16
PCL [ 22 ]	83.53	73.61	96.18	77.20	82.63

TABLE XIV: Integration of the proposed GCDG into DG methods on PACS.

Model	# of Params (M)	GFlops	Time (ms)
ERM	11.180	1.82167	19.727
MLP-ERM	11.212	1.82170	20.183
GCDG	11.188	1.82167	20.764

TABLE XV: Comparison of computational efficiency. MLP-ERM denotes ERM with MLP-based classifier. Tested with the image size of

224\times 224

on one NVIDIA Tesla V100 GPU.

Explore similar work

CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization
May 6, 2026Yu-Hsi Chen, Abd-Krim SeghouaneMultimodal Domain Generalization Subspace
The University of Melbourne
One-Step Generalization Ratio Guided Optimization for Domain Generalization
Jun 15, 2026Sumin Cho, Dongwon Kim, Kwangsu KimValue Alignment Overconfidence
Department of Computer Science and Engineering, University of Sungkyunkwan, Suwon, Korea.
Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts
Jun 24, 2026Tien-Hung Nguyen, Tien-Dat Tran, M. -Duong Nguyen +1Multimodal Domain Generalization Mixture-Of-Experts Architectures
VinUniversity, Vietnam

May 6, 2026, cs.CVJ/K move · Enter open · S save

CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization

Yu-Hsi Chen, Abd-Krim Seghouane

The University of Melbourne

Multimodal Domain Generalization Subspace Principal Component Analysis

Open paper AskPDF arXiv

Domain Generalization (DG) aims to learn representations that remain robust under out-of-distribution (OOD) shifts and generalize effectively to unseen target domains. While recent invariant learning strategies and architectural advances have achieved strong performance, explicitly discovering a structured domain-invariant subspace through second-order statistics remains underexplored. In this work, we propose CPCANet, a novel framework grounded in Common Principal Component Analysis (CPCA), which unrolls the iterative Flury-Gautschi (FG) algorithm into fully differentiable neural layers. This approach integrates the statistical properties of CPCA into an end-to-end trainable framework, enforcing the discovery of a shared subspace across diverse domains while preserving interpretability. Experiments on four standard DG benchmarks demonstrate that CPCANet achieves state-of-the-art (SOTA) performance in zero-shot transfer. Moreover, CPCANet is architecture-agnostic and requires no dataset-specific tuning, providing a simple and efficient approach to learning robust representations under distribution shift. Code is available at https://github.com/wish44165/CPCANet.

Abstract

Explore similar work

CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization

One-Step Generalization Ratio Guided Optimization for Domain Generalization

Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts