Low Dimensional Data

Low-dimensional data analysis focuses on extracting meaningful information from datasets where the underlying structure resides in a space of fewer dimensions than the observed data. Current research emphasizes developing efficient dimensionality reduction techniques, including those leveraging language models and various machine learning methods like survival models, and exploring the impact of low dimensionality on model performance and training dynamics, particularly concerning issues like dimensional collapse and spectral bias. These advancements are crucial for improving the efficiency and accuracy of analyses across diverse fields, from predicting real-world events using search data to optimizing high-dimensional functions in engineering and improving medical image analysis.

Papers

November 11, 2024

Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
Alex Havrilla, Wenjing Liao
Scaling Law Transformer Based Large Language Model Statistical Query Intrinsic Dimension Approximation Theory Low Dimensional Data Training Dataset Size Scaling Transformer

October 28, 2024

A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data
Saptarshi Chakraborty, Peter L. Bartlett
Generalization Error Generalization Property Intrinsic Dimension Statistical Analysis Federated Deep Low Dimensional Data Federated Learning Model

October 24, 2024

Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality
Zhihan Huang, Yuting Wei, Yuxin Chen
Generative AI Diffusion Explainer Probabilistic Model Intrinsic Dimension Iteration Complexity Low Dimensional Data

June 24, 2024

Compressing Search with Language Models
Thomas Mulc, Jennifer L. Steele
Language Model Large Scale Search Categorical Feature Low Dimensional Data

June 6, 2024

A Large-Scale Neutral Comparison Study of Survival Models on Low-Dimensional Data
Lukas Burk, John Zobolas, Bernd Bischl, Andreas Bender, Marvin N. Wright, Raphael Sonabend
Benchmark Platform Survival Model Low Dimensional Data Random Survival Forest Right Censored Large Scale Comparison

January 30, 2024

Multi-modal Representation Learning for Cross-modal Prediction of Continuous Weather Patterns from Discrete Low-Dimensional Data
Alif Bin Abdul Qayyum, Xihaier Luo, Nathan M. Urban, Xiaoning Qian, Byung-Jun Yoon
Dimension Reduction Multi Modal Representation Learning Climate Pattern Wind Data Low Dimensional Data Resolution Wind Cross Modal Prediction

January 19, 2024

LDReg: Local Dimensionality Regularized Self-Supervised Learning
Hanxun Huang, Ricardo J. G. B. Campello, Sarah Monazam Erfani, Xingjun Ma, Michael E. Houle, James Bailey
Self Supervised Deep Subspace Similarity Regularization Local Intrinsic Low Dimensional Data

June 17, 2023

Linearly-scalable learning of smooth low-dimensional patterns with permutation-aided entropic dimension reduction
Illia Horenko, Lukas Pospisil
Dimension Reduction Scalable Learning Low Dimensional Data Dimensional Data Pattern

February 14, 2023

Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
Minshuo Chen, Kaixuan Huang, Tuo Zhao, Mengdi Wang
Diffusion Model Estimation Task DIstribution Estimation Sample Complexity Bound Score Estimation Low Dimensional Data Dimensional Subspace

November 2, 2022

Linear Embedding-based High-dimensional Batch Bayesian Optimization without Reconstruction Mappings
Shuhei A. Horiguchi, Tomoharu Iwata, Taku Tsuzuki, Yosuke Ozawa
High Dimensional Bayesian Optimization Low Dimensional Data Batch Bayesian Optimization Dimensional Benchmark

March 7, 2022

Clustering and classification of low-dimensional data in explicit feature map domain: intraoperative pixel-wise diagnosis of adenocarcinoma of a colon in a liver
Dario Sitnik, Ivica Kopriva
Classification Code Inherent Interpretability Linear Classifier Low Dimensional Data Surgical Scene Segmentation Explicit Feature Map

March 1, 2022

Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space
Juncai He, Richard Tsai, Rachel Ward
LeArning Abstract Low Dimensional Manifold Euclidean Space Manifold Hypothesis Regularization Effect Side Effect Low Dimensional Data High Dimensional Euclidean Space

February 7, 2022

Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data
Umberto M. Tomasini, Antonio Sclocchi, Matthieu Wyart
Case Relevance Generalization Error Kernel Ridge Regression Spectral Bias Software Failure Laplacian Kernel Low Dimensional Data

December 13, 2021

Low Dimensional Data

Papers

Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data

A Statistical Analysis of Deep Federated Learning for Intrinsically Low-dimensional Data

Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality

Compressing Search with Language Models

A Large-Scale Neutral Comparison Study of Survival Models on Low-Dimensional Data

Multi-modal Representation Learning for Cross-modal Prediction of Continuous Weather Patterns from Discrete Low-Dimensional Data

LDReg: Local Dimensionality Regularized Self-Supervised Learning

Linearly-scalable learning of smooth low-dimensional patterns with permutation-aided entropic dimension reduction

Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data

Linear Embedding-based High-dimensional Batch Bayesian Optimization without Reconstruction Mappings

Clustering and classification of low-dimensional data in explicit feature map domain: intraoperative pixel-wise diagnosis of adenocarcinoma of a colon in a liver

Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space

Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data

Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias

Fast Single-Core K-Nearest Neighbor Graph Computation