Sparse Embeddings

Sparse embeddings aim to represent data using vectors with a minimal number of non-zero elements, improving efficiency and interpretability while preserving essential information. Current research focuses on developing algorithms to generate these sparse representations, including techniques like Independent Component Analysis (ICA) for enhanced interpretability and random matrix methods for efficient dimensionality reduction, often within the context of specific applications such as graph representation learning and information retrieval. The resulting sparse representations offer advantages in computational speed, memory usage, and model interpretability across diverse fields, impacting areas like medical diagnosis, natural language processing, and large-scale data analysis.

Papers

June 16, 2024

Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings
Hiroaki Yamagiwa, Momose Oyama, Hidetoshi Shimodaira
Jina Embeddings Normalization Dictionary Independent Component Analysis Cosine Similarity Sparse Embeddings Optimal Embeddings

November 17, 2023

Optimal Embedding Dimension for Sparse Subspace Embeddings
Shabarish Chenakkod, Michał Dereziński, Xiaoyu Dong, Mark Rudelson
Low Rank Approximation Box Dimension Sparse Embeddings

July 10, 2023

Learning Large Margin Sparse Embeddings for Open Set Medical Diagnosis
Mingyuan Liu, Lu Xu, Jicong Zhang
Sparse Data Local Descriptor Intra Class Class Separability Disease Detection Model Margin Loss Sparse Embeddings

June 9, 2023

Expectation-Complete Graph Representations with Homomorphisms
Pascal Welke, Maximilian Thiessen, Fabian Jogl, Thomas Gärtner
Graph Embeddings Graph Isomorphism Learning Benchmark Sparse Embeddings

April 3, 2023

Learning Sparsity of Representations with Discrete Latent Variables
Zhao Xu, Daniel Onoro Rubio, Giuseppe Serra, Mathias Niepert
Meaningful Representation Sparse Representation Sparse Structure Discrete Latent Variable Sparse Embeddings Sparsity Aware

February 13, 2023

Sparse Dimensionality Reduction Revisited
Mikael Møller Høgsgaard, Lion Kamma, Kasper Green Larsen, Jelani Nelson, Chris Schwiegelshohn
Sparsity Increase Sparse Principal Component Analysis Sparse Embeddings Johnson Lindenstrauss Transform

November 28, 2022

Unsupervised Superpixel Generation using Edge-Sparse Embedding
Jakob Geusen, Gustav Bredell, Tianfei Zhou, Ender Konukoglu
Adjacent Superpixels Superpixel Method Sparse Embeddings Pixel Embeddings

September 1, 2022

Johnson-Lindenstrauss embeddings for noisy vectors -- taking advantage of the noise
Zhen Shao
Industrial Disturbing Noise Advantage Feedback Sparse Embeddings Euclidean Embeddings Dense Embeddings High Dimensional Noisy Johnson Lindenstrauss

August 9, 2022

Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit
Mughilan Muthupari, Samrat Halder, Asad Sayeed, Yuval Marton
LeArning Abstract Representation Learning Jina Embeddings Word Embeddings Case Relevance Topic Analysis Semantic Role Labeling Compositional Semantics Sparse Embeddings

February 22, 2022

Robust and Provable Guarantees for Sparse Random Embeddings
Maciej Skorski, Alessandro Temperoni, Martin Theobald
Sparsity Increase Feature Embeddings Non Asymptotic Sparse Matrix Provable Guarantee Sparse Embeddings

January 14, 2022

Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval
Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Yingxia Shao, Defu Lian, Chaozhuo Li, Hao Sun, Denvy Deng, Liangjie Zhang, Qi Zhang, Xing Xie
App to App Retrieval Potential Scalability Embedding Based Retrieval Sparse Embeddings Dense Embeddings