Learning Signal

Learning signals are crucial for training artificial neural networks and understanding biological learning mechanisms, focusing on how networks effectively extract useful information from data to update their parameters. Current research investigates optimizing learning signals across various model architectures, including recurrent neural networks and graph neural networks, and explores alternative learning paradigms beyond backpropagation, such as equilibrium propagation and signal propagation, to improve efficiency and robustness. These advancements aim to enhance the performance and generalization capabilities of machine learning models, while also providing insights into the fundamental principles of learning in biological systems and informing the design of more efficient and biologically plausible artificial intelligence.

Papers

October 31, 2024

Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications
Matilda Knierim, Sahil Jain, Murat Han Aydoğan, Kenneth Mitra, Kush Desai, Akanksha Saran, Kim Baraka
Prosodic Feature Exploratory Study Algorithmic Decision Social Cue Interactive Reinforcement Learning Agent Learning Human Agent Interaction Learning Signal

August 24, 2023

Persistent learning signals and working memory without continuous attractors
Il Memming Park, Ábel Ságodi, Piotr Aleksander Sokół
Memory Trace Neural Dynamic Continuous Attractor Learning Signal

July 25, 2023

How to Scale Your EMA
Dan Busbridge, Jason Ramapuram, Pierre Ablin, Tatiana Likhomanenko, Eeshan Gunesh Dhekane, Xavier Suau, Russ Webb
Stochastic Gradient Descent Exponential Moving Average Machine Learning Tool Learning Signal

June 24, 2023

Graph Neural Networks Provably Benefit from Structural Information: A Feature Learning Perspective
Wei Huang, Yuan Cao, Haonan Wang, Xin Cao, Taiji Suzuki
Convolutional Neural Network Graph Representation Learning Graph Convolution Structural Information Learning Signal Two Layer

March 31, 2023

Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
Xuran Meng, Yuan Cao, Difan Zou
Gradient Descent Noisy Data Regularization Technique Gradient Regularization Standard Gradient Descent Label Regularization Learning Signal

November 15, 2022

Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
Chao Tao, Ji Qi, Mingning Guo, Qing Zhu, Haifeng Li
Technical Challenge Feature Learning Unsupervised Domain Learning Paradigm Self Supervised Feature Learning Signal Future Work

September 1, 2022

Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations
Axel Laborieux, Friedemann Zenke
Neuromorphic Computing Equilibrium Propagation Neural Oscillation Learning Signal Finite Size

July 4, 2022

The least-control principle for local learning at equilibrium
Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento
Optimal Control Neural Computation Deep Equilibrium Model Local Learning Equilibrium State Learning Signal Equilibrium Design Recurrent Equilibrium Network

May 27, 2022

AANG: Automating Auxiliary Learning
Lucio M. Dery, Paul Michel, Mikhail Khodak, Graham Neubig, Ameet Talwalkar
Auxiliary Task Auxiliary Learning Auxiliary Objective Avatar Animation Learning Signal

April 4, 2022

Signal Propagation: A Framework for Learning and Inference In a Forward Pass
Adam Kohan, Edward A. Rietman, Hava T. Siegelmann
LeArning Abstract New Framework Scientific Inference Back Propagation Propagation Environment Forward Pas Backward Learning Learning Signal Hardware Efficient

January 17, 2022

Distillation from heterogeneous unlabeled collections
Jean-Michel Begon, Pierre Geurts
Deep Network Linear Compression Mutual Distillation Image Classification Task Teacher Network Learning Signal

November 10, 2021

Structure from Silence: Learning Scene Structure from Ambient Sound
Ziyang Chen, Xixi Hu, Andrew Owens
Multimodal Model Inner Structure Multimodal Representation Multimodal Feature Ambient Sound Scene Parsing Learning Signal