cs.LG · 2607.16528 Copy arXiv ID · Jul 17, 2026 Save Hierarchical Domain Generalization Authors: Chenxiao Yang , Zhiyuan Li , Shai Ben-David , Nathan Srebro
Organizations: Toyota Technological Institute at Chicago · University of Waterloo
Abstract We study hierarchical domain generalization as a problem of extrapolation from finite observed regions to an entire instance space, replacing i.i.d. sampling with arbitrary domain hierarchies. We show that the central obstruction is not only the complexity of the hypothesis class, but the train/test domain partition through which evidence is revealed. In particular, no matter how small the class or how large the training size, some partition makes generalization fail for some target. These results suggest that modern generalization theory must treat domain structure as a first-class object.
Explore similar work Jun 24, 2026 · Tien-Hung Nguyen, Tien-Dat Tran, M. -Duong Nguyen +1 Multimodal Domain Generalization Domain Generalization
Jul 29, 2026 · Yuhang Jiang, Fengchuan Zhang, Sanguo Zhang +1 Multimodal Domain Generalization Domain Generalization
Jun 24, 2026 · cs.LG J/K move · Enter open · S save
Tien-Hung Nguyen, Tien-Dat Tran, M. -Duong Nguyen, Kok-Seng Wong
VinUniversity, Vietnam
Domain generalization (DG) aims to learn a model from one or more source domains that generalizes to an unseen target domain without accessing target data during training. A common approach enforces invariance of representations across all source domains, assuming predictive structure is globally shared. However, we demonstrate that enforcing invariance across more domains gradually restricts the feasible representation space, discarding transferable predictive factors that are not universally shared. To address this limitation, we propose subset-shared invariance, where predictive structure is assumed stable only within domain subsets. We implement this principle with a mixture-of-experts architecture, where each expert aligns the specific domains it serves and a routing mechanism composes subset-invariant components for prediction. This creates a routing-conditioned invariance, jointly learned with the representation. To facilitate effective decomposition, we develop training objectives that encourage selective alignment, confident and balanced routing, and diverse expert specialization. Experiments on DomainBed benchmarks demonstrate improved out-of-domain generalization and greater robustness under increasing domain heterogeneity. Our results suggest that DG should move beyond enforcing a single global invariance and instead model invariance through partially shared structure across domain subsets.