Local Adaptivity

Local adaptivity in machine learning focuses on developing algorithms and models that can efficiently adjust their behavior to specific, localized characteristics of data or environments, improving performance and robustness. Current research emphasizes techniques like parameter-efficient fine-tuning (e.g., LoRA), adaptive optimization algorithms (e.g., Adam with EMA), and graph-based methods for handling diverse data distributions and limited resources, particularly in federated learning settings. This research is significant because it addresses challenges in scalability, data heterogeneity, and privacy, leading to more efficient and effective models across diverse applications, including weather forecasting, robot learning, and natural language processing.

Papers

November 13, 2024

Noisy image decomposition: a new structure, texture and noise model based on local adaptivity
Jerome Gilles
Practical Algorithm Noisy Image Texture Rather Noise Model Local Adaptivity Image Decomposition Texture Quality

October 18, 2024

Unlabeled Action Quality Assessment Based on Multi-dimensional Adaptive Constrained Dynamic Time Warping
Renguang Chen, Guolong Zheng, Xu Yang, Zhide Chen, Jiwu Shu, Wencheng Yang, Kexin Zhu, Chen Feng
Dynamic Time Warping Action Quality Assessment Local Adaptivity Course Evaluation

October 10, 2024

Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity
Shuo Xie, Mohamad Amin Mohamadi, Zhiyuan Li
Language Model Convergence Analysis Loss Landscape Local Adaptivity

October 3, 2024

Task-unaware Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation
Pengzhi Yang, Xinyu Wang, Ruipeng Zhang, Cong Wang, Frans Oliehoek, Jens Kober
Anti Forgetting Retrieval Augmented Experience Replay Episodic Memory Local Adaptivity Lifelong Robot

September 11, 2024

Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region
Muhammad Akhtar Munir, Fahad Shahbaz Khan, Salman Khan
Case Study Numerical Weather Prediction Forecasting Performance Weather Forecasting Climate Model Local Adaptivity Area Modeling Neural Weather Model

August 30, 2024

Sparse Uncertainty-Informed Sampling from Federated Streaming Data
Manuel Röder, Frank-Michael Schleif
Many Sparse Data Stream Local Adaptivity Federated Data Uncertainty Sampling Label Extraction Front Diverse Batch

May 28, 2024

Adam with model exponential moving average is effective for nonconvex optimization
Kwangjun Ahn, Ashok Cutkosky
Nonconvex Optimization Exponential Mechanism Optimal Convergence Rate Local Adaptivity Adaptive Optimization Method Average Performance

March 13, 2024

On the Convergence of Locally Adaptive and Scalable Diffusion-Based Sampling Methods for Deep Bayesian Neural Network Posteriors
Tim Rensmeyer, Oliver Niggemann
Uncertainty Quantification Early Stage Convergence Bayesian Neural Network Markov Chain Monte Carlo Posterior Inference Diffusion Sampling Uncertainty Modeling Local Adaptivity

October 2, 2023

Sample-Efficiency in Multi-Batch Reinforcement Learning: The Need for Dimension-Dependent Adaptivity
Emmeran Johnson, Ciara Pike-Burke, Patrick Rebeschini
Reinforcement Learning Offline Reinforcement Learning Adaptive Importance Community Need Sample Efficiency Sample Efficient Local Adaptivity Batch Reinforcement Learning

August 14, 2023

Locally Adaptive and Differentiable Regression
Mingxuan Han, Varun Shankar, Jeff M Phillips, Chenglong Ye
Convergence Analysis Local Model Local Adaptivity Differentiable Model

June 12, 2023

Localised Adaptive Spatial-Temporal Graph Neural Network
Wenying Duan, Xiaoxi He, Zimu Zhou, Lothar Thiele, Hong Rao
Temporal Graph Temporal Graph Neural Network Spatial Temporal Graph Adjacency Matrix Graph Sparsification Local Adaptivity Spatial Graph

April 18, 2023

Romanization-based Large-scale Adaptation of Multilingual Language Models
Sukannya Purkayastha, Sebastian Ruder, Jonas Pfeiffer, Iryna Gurevych, Ivan Vulić
Language Model Cross Lingual Transfer Multilingual Language Model Local Adaptivity Machine Transliteration

December 20, 2022

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods
Zhuo Zhang, Yuanhang Yang, Yong Dai, Lizhen Qu, Zenglin Xu
Natural Language Processing Pre Trained Language Model Local Adaptivity Federated Large Language Model Federated Tuning

October 12, 2022

LACV-Net: Semantic Segmentation of Large-Scale Point Cloud Scene via Local Adaptive and Comprehensive VLAD
Ziyin Zeng, Yongyang Xu, Zhong Xie, Wei Tang, Jie Wan, Weichao Wu
Semantic Segmentation 3D Computer Vision Local Adaptivity Large Scale Point Cloud

August 21, 2022

Provable Adaptivity of Adam under Non-uniform Smoothness
Bohan Wang, Yushun Zhang, Huishuai Zhang, Qi Meng, Ruoyu Sun, Zhi-Ming Ma, Tie-Yan Liu, Zhi-Quan Luo, Wei Chen
Gradient Norm Non Uniform Fast Convergence Local Adaptivity Smoothness Assumption

July 18, 2022

Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks
Yabo Zhang, Mingshuai Yao, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Wangmeng Zuo
Generative Adversarial Network Zero Shot Diverse Set Shot Adaptation Local Adaptivity One Shot Unsupervised Domain Adaptation Level Adaptation

July 8, 2022

Graph-based Multi-View Fusion and Local Adaptation: Mitigating Within-Household Confusability for Speaker Identification
Long Chen, Yixiong Meng, Venkatesh Ravichandran, Andreas Stolcke
Semi Supervised Multi View Speaker Identification Voxceleb2 Dataset Local Adaptivity Heterogeneous User Speaker Label

June 15, 2022

ARES: Locally Adaptive Reconstruction-based Anomaly Scoring
Adam Goodge, Bryan Hooi, See Kiong Ng, Wee Siong Ng
Supervised Autoencoder Adaptive Importance Anomaly Score Anomaly Detection Performance Local Adaptivity

May 31, 2022

Minimax Classification under Concept Drift with Multidimensional Adaptation and Performance Guarantees
Verónica Álvarez, Santiago Mazuelas, Jose A. Lozano
Time Series Time Varying Concept Drift Performance Guarantee Local Adaptivity Supervised Classification Instance Level Pseudo Label Minimax Risk Classifier

April 25, 2022

Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods
Yi Wan, Ali Rahimi-Kalahroudi, Janarthanan Rajendran, Ida Momennejad, Sarath Chandar, Harm van Seijen
Model Based Reinforcement Learning Model Based Deep Learning Local Adaptivity Model Based Deep Reinforcement Learning Second Order Method

Local Adaptivity

Papers

Noisy image decomposition: a new structure, texture and noise model based on local adaptivity

Unlabeled Action Quality Assessment Based on Multi-dimensional Adaptive Constrained Dynamic Time Warping

Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity

Task-unaware Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation

Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region

Sparse Uncertainty-Informed Sampling from Federated Streaming Data

Adam with model exponential moving average is effective for nonconvex optimization

On the Convergence of Locally Adaptive and Scalable Diffusion-Based Sampling Methods for Deep Bayesian Neural Network Posteriors

Sample-Efficiency in Multi-Batch Reinforcement Learning: The Need for Dimension-Dependent Adaptivity

Locally Adaptive and Differentiable Regression

Localised Adaptive Spatial-Temporal Graph Neural Network

Romanization-based Large-scale Adaptation of Multilingual Language Models

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

LACV-Net: Semantic Segmentation of Large-Scale Point Cloud Scene via Local Adaptive and Comprehensive VLAD

Provable Adaptivity of Adam under Non-uniform Smoothness

Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks

Graph-based Multi-View Fusion and Local Adaptation: Mitigating Within-Household Confusability for Speaker Identification

ARES: Locally Adaptive Reconstruction-based Anomaly Scoring

Minimax Classification under Concept Drift with Multidimensional Adaptation and Performance Guarantees

Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods