Cross Distribution Generalization

Cross-distribution generalization in machine learning focuses on developing models that perform well on data drawn from distributions unseen during training, a crucial challenge for real-world applications. Current research investigates this challenge across diverse domains, including natural language processing (using instruction tuning and large language models), vehicle routing problems (employing knowledge distillation and distributionally robust optimization), and computer vision (specifically stereo matching). These studies highlight the need for robust model architectures and training strategies that go beyond simple memorization of training data, leading to more reliable and adaptable AI systems. Improved cross-distribution generalization promises significant advancements in various fields by enabling more robust and generalizable AI solutions.

Papers

October 30, 2023

Dynamics of Instruction Tuning: Each Ability of Large Language Models Has Its Own Growth Pace
Chiyu Song, Zhanchao Zhou, Jianhao Yan, Yuejiao Fei, Zhenzhong Lan, Yue Zhang
Instruction Tuning Critique Ability Instruction Tuned Model Instruction Data Cross Distribution Generalization

October 14, 2022

Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillation
Jieyi Bi, Yining Ma, Jiahai Wang, Zhiguang Cao, Jinbiao Chen, Yuan Sun, Yeow Meng Chee
Deep Learning Knowledge Distillation Distribution Sample Vehicle Routing Problem Learning Generalizable Cross Distribution Generalization

September 17, 2022

Active-Passive SimStereo -- Benchmarking the Cross-Generalization Capabilities of Deep Learning-based Stereo Methods
Laurent Jospin, Allen Antony, Lian Xu, Hamid Laga, Farid Boussaid, Mohammed Bennamoun
Active Stereo Learning Based Stereo Cross Distribution Generalization

February 15, 2022

Learning to Solve Routing Problems via Distributionally Robust Optimization
Yuan Jiang, Yaoxin Wu, Zhiguang Cao, Jie Zhang
LeArning Abstract Deep Learning Model Deep Model Robust Optimization Distributionally Robust Optimization Routing Problem Cross Distribution Generalization

Cross Distribution Generalization

Papers

Dynamics of Instruction Tuning: Each Ability of Large Language Models Has Its Own Growth Pace

Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillation

Active-Passive SimStereo -- Benchmarking the Cross-Generalization Capabilities of Deep Learning-based Stereo Methods

Learning to Solve Routing Problems via Distributionally Robust Optimization