Gradient Based

Gradient-based methods are central to training and interpreting many machine learning models, aiming to optimize model parameters and understand their decision-making processes. Current research focuses on improving the efficiency and robustness of gradient-based optimization, particularly within federated learning, and developing novel gradient-informed sampling techniques for enhanced model performance and explainability. These advancements are crucial for scaling machine learning to larger datasets and more complex tasks, impacting fields ranging from medical image analysis to natural language processing and optimization problems.

Papers

October 6, 2023

Accelerating optimization over the space of probability measures
Shi Chen, Qin Li, Oliver Tse, Stephen J. Wright
Optimization Purpose Gradient Based Deep Space Momentum Based Accelerated Gradient Method Probability Measure Hamiltonian Flow

October 4, 2023

ViT-ReciproCAM: Gradient and Attention-Free Visual Explanations for Vision Transformer
Seok-Yong Byun, Wonju Lee
Convolutional Neural Network Vision Transformer Computer Vision Natural Gradient Gradient Based Visual Explanation

October 2, 2023

September 7, 2023

Gradient-Based Feature Learning under Structured Data
Alireza Mousavi-Hosseini, Denny Wu, Taiji Suzuki, Murat A. Erdogdu
Gradient Based Batch Normalization Structured Data Spiked Matrix Shift Invariant Kernel

September 5, 2023

Learning Observation Models with Incremental Non-Differentiable Graph Optimizers in the Loop for Robotics State Estimation
Mohamad Qadri, Michael Kaess
Gradient Based State Estimation Robot Trajectory Graph Optimization Robotic Arm Activity Learning From Observation

August 26, 2023

Large-scale gradient-based training of Mixtures of Factor Analyzers
Alexander Gepperth
Mixture Component Gradient Based Gaussian Mixture Model Covariance Matrix High Dimensional Image Factor Analyzer

August 18, 2023

On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box
Yi Cai, Gerhard Wunder
Black Box Gradient Based Feature Attribution Attribution Method White Box Gradient Based Explanation

July 29, 2023

A Theory for Emergence of Complex Skills in Language Models
Sanjeev Arora, Anirudh Goyal
Language Model Theoretical Understanding Gradient Based Path Breaking Emergence Scaling Law Diverse Skill Generalization Theory

July 25, 2023

Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions
Skyler Wu, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, Himabindu Lakkaraju
Large Language Model Chain of Thought Gradient Based Feature Attribution Shot Prompting

July 18, 2023

July 15, 2023

Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversion
Lucas Böttcher
External Control Gradient Based Neural ODE Neural Ordinary Differential Equation System Identification Gradient Free Ensemble Kalman Inversion

July 9, 2023

Class-Incremental Mixture of Gaussians for Deep Continual Learning
Lukasz Korycki, Bartosz Krawczyk
Continual Learning Gradient Based RSD Difference of Gaussian Discriminative Feature COntinual Learning Framework Learning Mixture Continual Learning Model Time Varying Objective

July 8, 2023

Robust Ranking Explanations
Chao Chen, Chenghua Guo, Guixiang Ma, Ming Zeng, Xi Zhang, Sihong Xie
Adversarial Attack Adversarial Training Gradient Based Robust Explanation

July 6, 2023

Generalizing Backpropagation for Gradient-Based Interpretability
Kevin Du, Lucas Torroba Hennigen, Niklas Stoehr, Alexander Warstadt, Ryan Cotterell
Neural Network Deep Neural Network Back Propagation Gradient Flow Gradient Based Gradient Computation Network Gradient

July 1, 2023

Residual-based attention and connection to information bottleneck theory in PINNs
Sokratis J. Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, George Em Karniadakis
Physic Informed Neural Network Attention Mechanism Human Attention Neural Operator Gradient Based Information Bottleneck New Connection Gradient Weighting

June 29, 2023

Solving Kernel Ridge Regression with Gradient-Based Optimization Methods
Oskar Allerbo
Gradient Based Kernel Ridge Regression Ridge Regression Kernel Regression Kernel Flow

June 28, 2023

GraSS: Contrastive Learning with Gradient Guided Sampling Strategy for Remote Sensing Image Semantic Segmentation
Zhaoyang Zhang, Zhen Ren, Chao Tao, Yunsheng Zhang, Chengli Peng, Haifeng Li
Contrastive Learning Semantic Segmentation Contrastive Loss Gradient Based Contrastive Training Remote Sensing Image Semantic Segmentation Grass Growth

June 23, 2023

Four Axiomatic Characterizations of the Integrated Gradients Attribution Method
Daniel Lundstrom, Meisam Razaviyayn
Deep Neural Network Machine Learning Model Gradient Based Attribution Method Axiomatic Approach Integrated Gradient

Gradient Based

Papers

Accelerating optimization over the space of probability measures

ViT-ReciproCAM: Gradient and Attention-Free Visual Explanations for Vision Transformer

Intractability of Learning the Discrete Logarithm with Gradient-Based Methods

Nature Inspired Evolutionary Swarm Optimizers for Biomedical Image and Signal Processing -- A Systematic Review

Gradient-Based Feature Learning under Structured Data

Learning Observation Models with Incremental Non-Differentiable Graph Optimizers in the Loop for Robotics State Estimation

Large-scale gradient-based training of Mixtures of Factor Analyzers

On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box

A Theory for Emergence of Complex Skills in Language Models

Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimization

Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversion

Class-Incremental Mixture of Gaussians for Deep Continual Learning

Robust Ranking Explanations

Generalizing Backpropagation for Gradient-Based Interpretability

Residual-based attention and connection to information bottleneck theory in PINNs

Solving Kernel Ridge Regression with Gradient-Based Optimization Methods

GraSS: Contrastive Learning with Gradient Guided Sampling Strategy for Remote Sensing Image Semantic Segmentation

Four Axiomatic Characterizations of the Integrated Gradients Attribution Method