Stable Rank

Stable rank, a concept related to the effective dimensionality of data or model parameters, is central to various machine learning tasks, primarily aiming to improve efficiency and robustness of ranking algorithms. Current research focuses on applying stable rank concepts within diverse areas, including learning to rank for information retrieval, speech model evaluation, and active learning, often employing techniques like gradient boosted trees, neural networks, and variational autoencoders. These advancements offer improved performance and interpretability in ranking systems across numerous applications, from search engines and recommender systems to biomedical named entity recognition and even chiplet placement optimization in hardware design.

Papers

October 17, 2024

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
Chuanyu Tang, Yilong Chen, Zhenyu Zhang, Junyuan Shang, Wenyuan Zhang, Yong Huang, Tingwen Liu
Fine Tuning Mixture Component Mixture of Expert Low Rank Stable Rank

October 15, 2024

Offline Model-Based Optimization by Learning to Rank
Rong-Xi Tan, Ke Xue, Shen-Huan Lyu, Haopu Shang, Yao Wang, Yaoyuan Wang, Sheng Fu, Chao Qian
Surrogate Model Stable Rank Ranking Model Action Free Offline Better Optimizers Model Based Optimization

October 14, 2024

Lambda-Skip Connections: the architectural component that prevents Rank Collapse
Federico Arangath Joseph, Jerome Sieber, Melanie N. Zeilinger, Carmen Amo Alonso
Stable Rank Sequence Model Event Collapse Skip Connection Connected Component Rank Collapse

October 13, 2024

Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization
Alireza Salemi, Hamed Zamani
Large Language Model LeArning Abstract Retrieval Augmented Stable Rank Retrieval Augmentation Search Engine Knowledge Intensive Utility Maximization

October 7, 2024

ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillation
Yuelyu Ji, Zhuochun Li, Rui Meng, Daqing He
Stable Rank Ranking Model Document Re Ranking Text Ranking

September 16, 2024

Towards Automatic Assessment of Self-Supervised Speech Models using Rank
Zakaria Aldeneh, Vimal Thilak, Takuya Higuchi, Barry-John Theobald, Tatiana Likhomanenko
Stable Rank Self Supervised Speech Model Speech Supervised Learning Model Speech Domain Encoder Layer Automatic Assessment Unsupervised Metric

September 15, 2024

Proximal Ranking Policy Optimization for Practical Safety in Counterfactual Learning to Rank
Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Proximal Policy Optimization Human SAFETY Stable Rank Counterfactual Learning Inverse Propensity

August 28, 2024

Efficient LLM Scheduling by Learning to Rank
Yichao Fu, Siqi Zhu, Runlong Su, Aurick Qiao, Ion Stoica, Hao Zhang
Large Language Model Stable Rank Large Language Model Inference Dynamic Scheduling

August 23, 2024

Semi-Supervised Variational Adversarial Active Learning via Learning to Rank and Agreement-Based Pseudo Labeling
Zongyao Lyu, William J. Beksi
Active Learning Semi Supervised Variational Autoencoder Adversarial Learning Pseudo Labeling Stable Rank Domain Adversarial Semi Supervised Active Learning

August 22, 2024

Rank and Align: Towards Effective Source-free Graph Domain Adaptation
Junyu Luo, Zhiping Xiao, Yifan Wang, Xiao Luo, Jingyang Yuan, Wei Ju, Langechuan Liu, Ming Zhang
Stable Rank LD Align Graph Domain Adaptation Label Rich Source Invariant Subgraph Subgraph Extraction

August 19, 2024

Contextual Dual Learning Algorithm with Listwise Distillation for Unbiased Learning to Rank
Lulu Yu, Keping Bi, Shiyu Ni, Jiafeng Guo
Stable Rank Unbiased Learning Biased Feedback Dual Contrastive Learning Unbiased Learning to Rank Input Ranking Model

August 4, 2024

Understanding Deep Learning via Notions of Rank
Noam Razin
Deep Learning Strong Generalization Stable Rank Implicit Regularization Tensor Factorization

July 29, 2024

Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank
Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Proximal Policy Optimization Stable Rank Safety Guarantee Counterfactual Learning Inverse Propensity

July 23, 2024

On the Benefits of Rank in Attention Layers
Noah Amsel, Gilad Yehudai, Joan Bruna
Attention Mechanism Attention Layer Complementary Benefit Stable Rank Attention Head Attention Matrix Low Rank Attention

July 19, 2024

A3Rank: Augmentation Alignment Analysis for Prioritizing Overconfident Failing Samples for Deep Learning Models
Zhengyuan Wei, Haipeng Wang, Qilin Zhou, W. K. Chan
Deep Learning Model Stable Rank External Sample Based Rejection Low Confidence Augmentation Consistency

July 5, 2024

Revealing the Utilized Rank of Subspaces of Learning in Neural Networks
Isha Garg, Christian Koguchi, Eshan Verma, Daniel Ulbricht
Neural Network LeArning Abstract Supervised ImageNet Stable Rank Low Rank Decomposition Data Transformation Low Rank Structure

June 30, 2024

Sum-of-norms regularized Nonnegative Matrix Factorization
Andersen Ang, Waqas Bin Hamed, Hans De Sterck
Stable Rank Group Lasso

June 25, 2024

Learning to Rank for Maps at Airbnb
Malay Haldar, Hongwei Zhang, Kedar Bellare, Sherry Chen, Soumyadip Banerjee, Xiaotang Wang, Mustafa Abdool, Huiji Gao, Pavan Tapadia, Liwei He, Sanjeev Katariya
LeArning Abstract Partial Ranking Stable Rank Two Sided Digital Map Airbnb Rental Booking Probability

April 18, 2024

Estimating the Hessian Matrix of Ranking Objectives for Stochastic Learning to Rank with Gradient Boosted Trees
Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis
Hessian Matrix Stable Rank Learning to Rank Gradient Boosted Tree Probabilistic Ranking Stochastic Learning

April 7, 2024

Chiplet Placement Order Exploration Based on Learning to Rank with Graph Representation
Zhihui Deng, Yuanyuan Duan, Leilai Shao, Xiaolei Zhu
LeArning Abstract Graph Representation Stable Rank Chip Placement

Stable Rank

Papers

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning

Offline Model-Based Optimization by Learning to Rank

Lambda-Skip Connections: the architectural component that prevents Rank Collapse

Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization

ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillation

Towards Automatic Assessment of Self-Supervised Speech Models using Rank

Proximal Ranking Policy Optimization for Practical Safety in Counterfactual Learning to Rank

Efficient LLM Scheduling by Learning to Rank

Semi-Supervised Variational Adversarial Active Learning via Learning to Rank and Agreement-Based Pseudo Labeling

Rank and Align: Towards Effective Source-free Graph Domain Adaptation

Contextual Dual Learning Algorithm with Listwise Distillation for Unbiased Learning to Rank

Understanding Deep Learning via Notions of Rank

Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank

On the Benefits of Rank in Attention Layers

A3Rank: Augmentation Alignment Analysis for Prioritizing Overconfident Failing Samples for Deep Learning Models

Revealing the Utilized Rank of Subspaces of Learning in Neural Networks

Sum-of-norms regularized Nonnegative Matrix Factorization

Learning to Rank for Maps at Airbnb

Estimating the Hessian Matrix of Ranking Objectives for Stochastic Learning to Rank with Gradient Boosted Trees

Chiplet Placement Order Exploration Based on Learning to Rank with Graph Representation