Divergence Loss

Divergence loss is a technique used in machine learning to measure and reduce the difference between probability distributions, primarily addressing challenges arising from data inconsistencies or distribution shifts. Current research focuses on applying divergence loss within various contexts, including test-time adaptation for improved model robustness and generalization across different datasets, unlearning in large language models to enhance privacy, and improving the performance of federated learning in scenarios with non-IID data. These advancements are significant because they enhance the reliability, adaptability, and privacy of machine learning models across diverse applications, particularly in medical image analysis and autonomous driving.

Papers

October 15, 2024

Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss
Yifei Zhang, Huan-ang Gao, Zhou Jiang, Hao Zhao
Optical Flow Turbulent Flow Particle Image Velocimetry Test Time Optimization Divergence Loss

October 9, 2024

Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching
Wenqi Niu, Yingchao Wang, Guohui Cai, Hanpo Hou
Knowledge Distillation High Efficiency Good Teacher CORrelation Matching Divergence Loss

May 21, 2024

Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models
Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, Sheng Bi
Multimodal Large Language Model Cross Entropy Loss Unlearning Framework Divergence Loss

May 14, 2024

Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation
Shishuai Hu, Zehui Liao, Zeyou Liu, Yong Xia
Medical Image Segmentation Test Time Adaptation Batch Normalization Time Adaptation Divergence Loss Patient Specific Segmentation

March 18, 2024

Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting
Mingkui Tan, Guohao Chen, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Peilin Zhao, Shuaicheng Niu
Test Time Adaptation Entropy Regularization Divergence Loss

August 29, 2023

Bridging Distribution Learning and Image Clustering in High-dimensional Space
Guanfang Dong, Chenqiu Zhao, Anup Basu
Data Clustering Gaussian Mixture Model High Dimensional Space Distribution Learning Distributional Learning Image Clustering Divergence Loss

May 23, 2023

Decoupled Kullback-Leibler Divergence Loss
Jiequan Cui, Zhuotao Tian, Zhisheng Zhong, Xiaojuan Qi, Bei Yu, Hanwang Zhang
Knowledge Distillation Adversarial Training Cross Entropy Loss Consistency Regularization Kullback Leibler Divergence Divergence Loss

March 11, 2023

Reducing Non-IID Effects in Federated Autonomous Driving with Contrastive Divergence Loss
Tuong Do, Binh X. Nguyen, Hien Nguyen, Erman Tjiputra, Quang D. Tran, Te-Chuan Chiu, Anh Nguyen
Autonomous Driving Mixed Effect Federated Edge Learning Contrastive Divergence Divergence Loss

September 23, 2022

Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks
Ponkrshnan Thiagarajan, Susanta Ghosh
Loss Function Bayesian Neural Network Kullback Leibler Divergence Jensen Shannon Divergence Divergence Loss

June 2, 2022

Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction
Xiangyuan Yang, Jie Lin, Hanlin Zhang, Xinyu Yang, Peng Zhao
Deep Neural Network Native Robustness Strong Generalization Adversarial Example Adversarial Training Loss Function Better Robustness Confidence Aware Divergence Loss

Divergence Loss

Papers

Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss

Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching

Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models

Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation

Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting

Bridging Distribution Learning and Image Clustering in High-dimensional Space

Decoupled Kullback-Leibler Divergence Loss

Reducing Non-IID Effects in Federated Autonomous Driving with Contrastive Divergence Loss

Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks

Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction