Dictionary Learning

Dictionary learning is a machine learning technique focused on decomposing complex data into a set of simpler, interpretable components, or "atoms," forming a dictionary. Current research emphasizes developing robust algorithms, such as sparse autoencoders and variations of iterative shrinkage-thresholding algorithms (ISTA), to learn these dictionaries effectively, particularly in high-dimensional spaces and under noisy conditions. Applications span diverse fields, including signal processing, image analysis, and natural language processing, where dictionary learning aids in feature extraction, anomaly detection, and model interpretability, ultimately improving the efficiency and understanding of complex systems.

Papers

November 15, 2024

Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning
Jeffrey Olmo, Jared Wilson, Max Forsey, Bryce Hepner, Thomas Vin Howe, David Wingate
Supervised Autoencoder Feature Wise Large Relevance Improvement Qualitative Difference Dictionary Learning Sparse Autoencoders Sparse Auto Encoders

October 31, 2024

Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learning
John Wu, David Wu, Jimeng Sun
Language Model Transparency Index Dictionary Learning Medical Coding Label Attention Medical Code Prediction

October 10, 2024

Efficient Dictionary Learning with Switch Sparse Autoencoders
Anish Mudide, Joshua Engels, Eric J. Michaud, Max Tegmark, Christian Schroeder de Witt
Sparsity Increase Dictionary Learning Sparse Autoencoders Sparse Mixture

July 31, 2024

Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models
Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, Samuel Marks
Language Model Much Progress Latent Feature Dictionary Learning Sparse Autoencoders Game Model Language Model Representation

July 26, 2024

A Model for Combinatorial Dictionary Learning and Inference
Avrim Blum, Kavya Ravichandran
Full Model Scientific Inference Low Rank Latent Feature Dictionary Learning Adversarial Corruption

July 9, 2024

Variational Learning ISTA
Fabio Valerio Massoli, Christos Louizos, Arash Behboodi
Sparsity Increase Compressed Sensing Sparse Representation Dictionary Learning Variational Learning

June 17, 2024

Unmixing Noise from Hawkes Process to Model Learned Physiological Events
Guillaume Staerman, Virginie Loison, Thomas Moreau
Full Model Event Detection Joint Learning Unstructured Text Physiological Signal Hawkes Process Dictionary Learning Object Based Spatial Unmixing Model Physiological State

May 17, 2024

Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning
Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, Lee Sharkey
Many Sparse Mechanistic Interpretability Dictionary Learning Sparse Autoencoders Important Feature

May 14, 2024

Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control
Aleksandar Makelov, George Lange, Neel Nanda
Inherent Interpretability External Control Dictionary Learning Interpretable Feature Sparse Autoencoders Feature Encoding

May 13, 2024

Efficient Matrix Factorization Via Householder Reflections
Anirudh Dash, Aditya Siripuram
Matrix Factorization Dictionary Learning Robust Recovery Low Rank Factorization

April 28, 2024

Lightweight Conceptual Dictionary Learning for Text Classification Using Information Compression
Li Wan, Tansu Alpcan, Margreta Kuijper, Emanuele Viterbo
Text Classification Data Compression Dictionary Learning Text Datasets

April 24, 2024

Improving Dictionary Learning with Gated Sparse Autoencoders
Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, Neel Nanda
Language Model Dictionary Learning Sparse Autoencoders

April 15, 2024

Dynamic fault detection and diagnosis of industrial alkaline water electrolyzer process with variational Bayesian dictionary learning
Qi Zhang, Lei Xie, Weihua Xu, Hongye Su
Medical Diagnosis Dictionary Learning Fuel Cell Sparse Bayesian Modular Electrolysis Plant

April 5, 2024

Fusing Dictionary Learning and Support Vector Machines for Unsupervised Anomaly Detection
Paul Irofti, Iulian-Andrei Hîji, Andrei Pătraşcu, Nicolae Cleju
Unsupervised Anomaly Detection Sparse Representation Dictionary Learning One Class Support Vector Machine Sparse Pattern

April 1, 2024

Decentralized Collaborative Learning Framework with External Privacy Leakage Analysis
Tsuyoshi Idé, Dzung T. Phan, Rudy Raymond
Decentralized Learning PRIvacy Leakage Anomaly Score Dictionary Learning Privacy Leakage

February 19, 2024

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
Zhengfu He, Xuyang Ge, Qiong Tang, Tianxiang Sun, Qinyuan Cheng, Xipeng Qiu
Case Study Mechanistic Interpretability Dictionary Learning Circuit Discovery Dictionary Based Fine Grained Complexity

December 13, 2023

Explainable Trajectory Representation through Dictionary Learning
Yuanbo Tang, Zhiyuan Peng, Yang Li
Dictionary Learning Trajectory Representation Local Trajectory Shape Descriptor Real World Taxi Trajectory Autoencoder

October 31, 2023

SC-MIL: Sparsely Coded Multiple Instance Learning for Whole Slide Image Classification
Peijie Qiu, Pan Xiao, Wenhui Zhu, Yalin Wang, Aristeidis Sotiras
Multiple Instance Learning Dictionary Learning Whole Slide Image Classification Instance Embeddings

October 29, 2023

Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming
Gregory Dexter, Petros Drineas, David P. Woodruff, Taisuke Yasuda
Practical Algorithm K$ Mean Dictionary Learning Streaming Data Polynomial Time Approximation

September 26, 2023

Zero-Shot Constrained Motion Planning Transformers Using Learned Sampling Dictionaries
Jacob J. Johnson, Ahmed H. Qureshi, Michael C. Yip
Dictionary Learning Feasible Trajectory Sampling Based Motion Planner Planning Transformer Constrained Motion Planning

Dictionary Learning

Papers

Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning

Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learning

Efficient Dictionary Learning with Switch Sparse Autoencoders

Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models

A Model for Combinatorial Dictionary Learning and Inference

Variational Learning ISTA

Unmixing Noise from Hawkes Process to Model Learned Physiological Events

Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning

Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control

Efficient Matrix Factorization Via Householder Reflections

Lightweight Conceptual Dictionary Learning for Text Classification Using Information Compression

Improving Dictionary Learning with Gated Sparse Autoencoders

Dynamic fault detection and diagnosis of industrial alkaline water electrolyzer process with variational Bayesian dictionary learning

Fusing Dictionary Learning and Support Vector Machines for Unsupervised Anomaly Detection

Decentralized Collaborative Learning Framework with External Privacy Leakage Analysis

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT

Explainable Trajectory Representation through Dictionary Learning

SC-MIL: Sparsely Coded Multiple Instance Learning for Whole Slide Image Classification

Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming

Zero-Shot Constrained Motion Planning Transformers Using Learned Sampling Dictionaries