Learning Rule

Learning rules govern how artificial neural networks and other machine learning models adapt and learn from data, aiming to optimize performance on a given task. Current research focuses on understanding the mathematical properties of various learning rules, including their relationship to gradient descent, their vulnerability to adversarial attacks (especially in spiking neural networks), and their impact on representation learning and network dynamics across different architectures. This research is crucial for improving the efficiency, robustness, and interpretability of machine learning models, with implications for diverse applications ranging from robotics and anomaly detection to knowledge graph completion and brain-computer interfaces.

Papers

January 25, 2022

Representations learnt by SGD and Adaptive learning rules: Conditions that vary sparsity and selectivity in neural networks
Jin Hyun Park
Neural Network Stochastic Gradient Descent Meaningful Representation Sparsity Increase Sparse Activation Selective Engagement Learning Rule Selective Neural Network

November 4, 2021

Model-Free Risk-Sensitive Reinforcement Learning
Grégoire Delétang, Jordi Grau-Moya, Markus Kunesch, Tim Genewein, Rob Brekelmans, Shane Legg, Pedro A. Ortega
Stochastic Approximation Learning Rule Risk Based Decision

Learning Rule

Papers

Representations learnt by SGD and Adaptive learning rules: Conditions that vary sparsity and selectivity in neural networks

Model-Free Risk-Sensitive Reinforcement Learning