Overview

Multi-index models represent a high-dimensional input through a small set of directions, then apply a nonlinear function to those projections; learning these directions can make prediction more sample-efficient and reveal useful features. Approaches include hyper-kernel ridge regression and gradient-based subspace recovery, neural-network SGD and gradient-flow analyses, and approximate message passing in high-dimensional Gaussian settings. Results study when methods recover the hidden subspace before achieving accurate prediction, and how symmetry, incremental feature learning, and flatness affect recovery and generalization. A central tension is that models may be statistically learnable yet computationally difficult: statistical-query limits do not always predict vanilla SGD, and success can depend on activation functions, initialization, noise, and model structure.

Latest papers

CardsList

14 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. Gaussian Partitioning for Reliable Learning of Multi-Index Models

    Nov 3, 2026Ilias Diakonikolas, Giannis Iakovidis, Daniel M. Kane +1Learning with Distribution ShiftAgnostic Learning

  2. Robust Learning of Multi-Index Models under Logconcave Distributions via Weak Partial Learners

    Oct 30, 2026Ilias Diakonikolas, Daniel M. Kane, Mingchen MaStatistical Learning TheoryClassification

  3. Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models

    Oct 6, 2026Jivan Waber, Vanessa Piccolo, Yatin Dandi +1Representation LearningData Augmentation

  4. Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model

    Jun 26, 2026Andrea Montanari, Kangjie ZhouApproximate Message PassingStatistical Learning Theory

  5. CAMI: Cost-Aware Agent-Guided Multi-Indexing for Semantic Retrieval

    Jun 14, 2026Adnan Qidwai, Anand Eswaran, Sonam Mishra +2Semantic SearchInformation Retrieval

  6. Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models

    May 14, 2026Libin Zhu, Damek Davis, Dmitriy Drusvyatskiy +1Kernel Ridge RegressionRepresentation Learning

  7. A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models

    Mar 18, 2026Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1Statistical Learning TheoryMulti-Index Models

  8. Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks

    Mar 16, 2026Yuri Kinoshita, Naoki Nishikawa, Taro ToyoizumiTwo-Layer Neural NetworksDataset Distillation

  9. Optimal scaling laws in learning hierarchical multi-index models

    Feb 5, 2026Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1Neural Scaling LawsTwo-Layer Neural Networks

  10. Limitations of SGD for Multi-Index Models Beyond Statistical Queries

    Feb 5, 2026Daniel Barzilai, Ohad ShamirSingle-Index ModelsStatistical Learning Theory

  11. Learning Multi-Index Models with Hyper-Kernel Ridge Regression

    Oct 2, 2025Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2Kernel Ridge RegressionKernel Regression

  12. Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension

    Jul 1, 2024Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis +2Agnostic LearningMargin-Based Learning