Organizations: Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China · College of Computer Science and Technology and the Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun, China
Domain generalization (DG) aims to improve the generalizability of computer vision models toward distribution shifts. The mainstream DG methods predominantly focus on learning domain invariance across domains, however, such methods overlook the untapped potential inherent in domain-specific information. While the prevailing practice of discriminative linear classifier has been tailored to domain-invariant features, it struggles when confronted with diverse domain-specific information, \emph{e.g.,} intra-class shifts, that exhibits multi-modality. To address these issues, we explore the theoretical implications of relying on domain-invariant features, revealing the crucial role of domain-specific information in mitigating the target risk for DG. Drawing from these insights, we propose Generative Classifier-driven Domain Generalization (GCDG), introducing a generative paradigm for the DG classifier based on Gaussian Mixture Models (GMMs) for each class across domains. GCDG consists of three key modules: Heterogeneity Learning Classifier~(HLC), Spurious Correlation Blocking~(SCB), and Diverse Component Balancing~(DCB). Concretely, HLC attempts to model the feature distributions and thereby capture valuable domain-specific information via GMMs. SCB identifies the neural units containing spurious correlations and perturbs them, mitigating the risk of HLC learning irrelevant spurious patterns. Meanwhile, DCB ensures a balanced contribution of components within HLC, preventing the underestimation or neglect of critical components. In this way, GCDG excels in capturing the nuances of domain-specific information characterized by diverse distributions. Extensive experiments show GCDG's comparable performance on five DG benchmarks and one face anti-spoofing dataset, seamlessly integrating into existing DG methods with consistent improvements.
Figures & tables
Fig. 1: Comparison of modeling a class between the discriminative linear classifier and the proposed generative classifier in DG. (a) The prevailing linear classifier in DG operates under the assumption of unimodal distribution, encountering substantial challenges when confronted with domain-specific data that exhibits multi-modality. (b) In this paper, we introduce a novel generative classifier to capture the underlying multi-modal distribution present in domain-specific data.
Fig. 2: The framework of our proposed GCDG. The key innovation is the Heterogeneity Learning Classifier, which is a generative classifier consisting of a mixture of Gaussians for each class and adept at effectively harnessing valuable domain-specific information exhibiting multi-modality. Besides, we introduce Spurious Correlation Blocking to shuffle the neural units containing spurious correlations, mitigating their adverse effect on capturing domain-specific information. Furthermore, Diverse Component Balancing is designed to balance the contributions of diverse components, avoiding underestimating essential ones.
Methods
OfficeHome ( ↑ )
DomainNet ( ↑ )
ERM
60.51
43.68
GMMSeg
60.16
13.16
GCDG (ours)
64.49
46.60
TABLE I: Comparison of the generalizability on datasets where the number of samples in one class is small.
Model
PACS ( ↑ )
VLCS ( ↑ )
OH ( ↑ )
TI ( ↑ )
DN ( ↑ )
Avg. ( ↑ )
SAM [ 84 ]
96.64
85.01
79.30
91.25
64.44
83.33
SWAD [ 85 ]
96.20
84.44
78.53
90.90
64.44
82.90
PCL [ 22 ]
96.17
84.16
79.60
87.89
64.25
82.41
GCDG (ours)
96.98
85.52
79.62
92.75
64.91
83.96
TABLE II: Comparisons of in-domain generalization on five DG benchmarks with flatness-aware optimization methods.
Methods
Office-Home (Clipart)
PACS (Cartoon)
Entropy
Accuracy
Entropy
Accuracy
ERM
7.04
48.00
7.96
74.79
GCDG
7.62
51.27
8.66
78.58
TABLE III: Comparison of the average entropy values of features on source domains when the model is converged.
Parameter
Value
Learning rate
[5e-5, 8e-5]
Number of components K
[2, 3, 5]
Compression dimension D
[64, 1024]
Quantile q
[10, 20, 30]
TABLE IV: Hyperparameter search space.
Method
Target domain
Avg.( ↑ )
Art
Cartoon
Photo
Sketch
ResNet-18
RSC [ 94 ] (ECCV’2020)
79.88
76.87
94.56
77.11
82.10
MTL [ 95 ] (JMLR’2021)
79.99
72.18
95.28
74.94
80.60
SagNet [ 96 ] (CVPR’2021)
81.15
75.05
94.61
75.38
81.55
ARM [ 97 ] (NeurIPS’2021)
80.42
75.96
95.21
72.33
80.98
TABLE V: Generalization results of state-of-the-art methods and our GCDG on PACS.
Method
Target domain
Avg.( ↑ )
L100
L38
L43
L46
ResNet-18
RSC [ 94 ] (ECCV’2020)
47.32
37.66
51.67
35.95
43.15
MTL [ 95 ] (JMLR’2021)
38.94
35.18
52.80
35.29
40.55
SagNet [ 96 ] (CVPR’2021)
47.25
29.67
52.87
25.22
38.75
ARM [ 97 ] (NeurIPS’2021)
44.98
33.73
43.39
27.77
37.47
TABLE VI: Performance comparison with state-of-the-art methods on Terra-Incognita.
Method
Target domain
Avg.( ↑ )
Art
Clipart
Product
Real
ResNet-18
RSC [ 94 ] (ECCV’2020)
49.38
45.91
66.84
67.41
57.38
MTL [ 95 ] (JMLR’2021)
52.58
46.99
70.83
72.46
60.72
SagNet [ 96 ] (CVPR’2021)
56.28
51.32
70.64
73.38
62.90
ARM [ 97 ] (NeurIPS’2021)
52.68
45.82
68.64
71.40
59.63
TABLE VII: Performance comparison with state-of-the-art approaches on Office-Home.
Method
Target domain
Avg.( ↑ )
Caltech
LabelMe
SUN
PASCAL
ResNet-18
RSC [ 94 ] (ECCV’2020)
93.29
64.47
71.52
73.31
75.65
MTL [ 95 ] (JMLR’2021)
96.38
62.54
70.91
71.68
75.38
SagNet [ 96 ] (CVPR’2021)
97.09
62.07
70.37
75.42
76.24
ARM [ 97 ] (NeurIPS’2021)
96.29
61.55
72.32
76.27
76.61
TABLE VIII: Generalization results of state-of-the-art methods and our GCDG on VLCS.
Methods
I&C&M to O
O&C&I to M
O&M&I to C
Avg.
HTER(%)
AUC(%)
HTER(%)
AUC(%)
HTER(%)
AUC(%)
HTER(%)
AUC(%)
MADDG [ 63 ] (CVPR’2019)
27.98
80.02
17.69
88.06
24.50
84.51
23.39
84.20
D 2 AM [ 103 ] (AAAI’2021)
15.27
90.87
12.70
95.66
20.98
85.58
16.32
90.70
SSDG [ 104 ] (CVPR’2020)
25.17
81.83
16.67
90.47
23.11
85.45
21.65
85.92
RFM [ 105 ] (AAAI’2020)
16.45
91.16
13.89
93.98
20.27
88.16
16.87
91.1
DRDG [ 106 ] (IJCAI’2021)
15.63
91.75
12.43
95.81
19.05
88.79
15.70
92.12
TABLE IX: Comparison to the state-of-the-art FAS methods. The bold numbers indicate the best performance.
Method
Target domain
Avg.( ↑ )
Clipart
Infograph
painting
Quickdraw
Real
Sketch
ResNet-50
RSC [ 94 ] (ECCV’2022)
55.0
18.3
44.4
12.2
55.7
47.8
38.9
MTL [ 95 ] (JMLR’2021)
57.9
18.5
46.0
12.5
59.5
49.2
40.6
SagNet [ 96 ] (CVPR’2021)
57.7
19.0
45.3
12.7
58.1
48.8
40.3
ARM [ 97 ] (NeurIPS’2021)
49.7
16.3
40.9
9.4
53.4
43.5
35.5
TABLE X: Performance comparison with state-of-the-art methods on DomainNet. † denotes reproduced results.
Model
PACS ( ↑ )
VLCS ( ↑ )
OH ( ↑ )
TI ( ↑ )
DN ( ↑ )
Avg. ( ↑ )
ERM
80.72
74.50
60.51
41.44
43.68
60.17
MLP-ERM
81.38
73.85
61.24
43.17
42.68
60.47
GCDG (ours)
83.44
76.86
64.49
44.86
46.63
63.26
TABLE XI: Generalization performance with various classifiers on five commonly used DG datasets. MLP-ERM denotes ERM with MLP-based classifier.
Fig. 4: Visualization of the loss landscapes for ERM, the flatness-aware method SWAD [ 85 ] , and the proposed GCDG on PACS. Note that the loss landscape is visualized on the source domains. Notably, our proposed GCDG exhibits superior efficacy in fostering flat minima compared to ERM and the flatness-aware method SWAD.
ID
HLC
DCB
SCB
Target domain
Avg.( ↑ )
Art
Cartoon
Photo
Sketch
A
-
-
-
78.76
74.79
96.29
73.02
80.72
B
✓
-
-
81.84
74.49
94.13
77.35
81.95
C
✓
✓
-
81.74
74.27
93.71
79.82
82.39
D
✓
-
✓
82.13
75.85
94.91
76.28
82.29
E
✓
✓
✓
83.06
78.50
92.63
79.56
83.44
TABLE XII: Ablation study of the proposed components in GCDG on PACS.
Models
Target domain
Avg.( ↑ )
Art
Cartoon
Photo
Sketch
DMG [ 35 ]
76.90
80.38
93.35
75.21
81.46
DDG [ 37 ]
79.30
74.00
91.80
75.80
80.20
DRM [ 68 ]
81.20
71.20
93.70
78.60
81.20
GCDG (ours)
83.06
78.50
92.63
79.56
83.44
TABLE XIII: Comparison between methods leveraging domain-specific information.
Models
Target domain
Avg.( ↑ )
Art
Cartoon
Photo
Sketch
ERM
78.76
74.79
96.29
73.02
80.72
+ GCDG
83.06
78.50
92.63
79.56
83.44
SWAD [ 85 ]
83.28
74.63
96.56
77.96
83.11
+ GCDG
85.17
78.04
95.14
78.28
84.16
PCL [ 22 ]
83.53
73.61
96.18
77.20
82.63
TABLE XIV: Integration of the proposed GCDG into DG methods on PACS.
Model
# of Params (M)
GFlops
Time (ms)
ERM
11.180
1.82167
19.727
MLP-ERM
11.212
1.82170
20.183
GCDG
11.188
1.82167
20.764
TABLE XV: Comparison of computational efficiency. MLP-ERM denotes ERM with MLP-based classifier. Tested with the image size of 224×224 on one NVIDIA Tesla V100 GPU.
Domain Generalization (DG) aims to learn representations that remain robust under out-of-distribution (OOD) shifts and generalize effectively to unseen target domains. While recent invariant learning strategies and architectural advances have achieved strong performance, explicitly discovering a structured domain-invariant subspace through second-order statistics remains underexplored. In this work, we propose CPCANet, a novel framework grounded in Common Principal Component Analysis (CPCA), which unrolls the iterative Flury-Gautschi (FG) algorithm into fully differentiable neural layers. This approach integrates the statistical properties of CPCA into an end-to-end trainable framework, enforcing the discovery of a shared subspace across diverse domains while preserving interpretability. Experiments on four standard DG benchmarks demonstrate that CPCANet achieves state-of-the-art (SOTA) performance in zero-shot transfer. Moreover, CPCANet is architecture-agnostic and requires no dataset-specific tuning, providing a simple and efficient approach to learning robust representations under distribution shift. Code is available at https://github.com/wish44165/CPCANet.
Domain Generalization (DG) aims to train models that generalize to unseen target domains but often overfit to domain-specific features, known as undesired correlations. Gradient-based DG methods typically guide gradients in a dominant direction but often inadvertently reinforce spurious correlations. Recent work has employed dropout to regularize overconfident parameters, but has not explicitly adjusted gradient alignment or ensured balanced parameter updates. We propose GENIE (Generalization-ENhancing Iterative Equalizer), a novel optimizer that leverages the One-Step Generalization Ratio (OSGR) to quantify each parameter's contribution to loss reduction and assess gradient alignment. By dynamically equalizing OSGR via a preconditioning factor, GENIE prevents a small subset of parameters from dominating optimization, thereby promoting domain-invariant feature learning. Theoretically, GENIE balances convergence contribution and gradient alignment among parameters, achieving higher OSGR while retaining SGD's convergence rate. Empirically, it outperforms existing optimizers and enhances performance when integrated with various DG and single-DG methods.
Sumin Cho, Dongwon Kim, Kwangsu Kim
Department of Computer Science and Engineering, University of Sungkyunkwan, Suwon, Korea.
Domain generalization (DG) aims to learn a model from one or more source domains that generalizes to an unseen target domain without accessing target data during training. A common approach enforces invariance of representations across all source domains, assuming predictive structure is globally shared. However, we demonstrate that enforcing invariance across more domains gradually restricts the feasible representation space, discarding transferable predictive factors that are not universally shared. To address this limitation, we propose subset-shared invariance, where predictive structure is assumed stable only within domain subsets. We implement this principle with a mixture-of-experts architecture, where each expert aligns the specific domains it serves and a routing mechanism composes subset-invariant components for prediction. This creates a routing-conditioned invariance, jointly learned with the representation. To facilitate effective decomposition, we develop training objectives that encourage selective alignment, confident and balanced routing, and diverse expert specialization. Experiments on DomainBed benchmarks demonstrate improved out-of-domain generalization and greater robustness under increasing domain heterogeneity. Our results suggest that DG should move beyond enforcing a single global invariance and instead model invariance through partially shared structure across domain subsets.
Tien-Hung Nguyen, Tien-Dat Tran, M. -Duong Nguyen +1