Gram Matrix

The Gram matrix, representing pairwise inner products of data vectors, is a fundamental tool in machine learning and signal processing, primarily used for analyzing data structure and relationships. Current research focuses on efficient computation and application of Gram matrices in diverse areas, including improving the robustness and training stability of convolutional neural networks (via spectral norm estimation and Lipschitz regularization), enhancing n-gram language models, and enabling unsupervised domain adaptation regression. These advancements have significant implications for various fields, offering improved performance in tasks such as image denoising, backdoor detection in deep learning models, and high-dimensional data clustering.

Papers

October 8, 2024

Simple Relative Deviation Bounds for Covariance and Gram Matrices
Daniel Barzilai, Ohad Shamir
Covariance Conditioning Smallest Eigenvalue Gram Matrix

January 31, 2024

Spectral Norm of Convolutional Layers with Circular and Zero Paddings
Blaise Delattre, Quentin Barthélemy, Alexandre Allauzen
Convolutional Layer Zero Padding Gram Matrix Circular Convolution Spectral Norm

January 30, 2024

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, Hannaneh Hajishirzi
Language Model K TOKEN Gram Matrix Gram Language Model

May 28, 2023

On the impact of activation and normalization in obtaining isometric embeddings at initialization
Amir Joudaki, Hadi Daneshmand, Francis Bach
Deep Neural Network Global Impact Normalization Dictionary New Initialization Targeted Activation Penalty Layer Normalization Normalization Layer Gram Matrix Isometric Representation

May 25, 2023

Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration
Blaise Delattre, Quentin Barthélemy, Alexandre Araujo, Alexandre Allauzen
Convolutional Layer Lower Bound Lipschitz Bound Superlinear Convergence Gram Matrix Spectral Norm Lipschitz Regularization Circulant Matrix

March 23, 2023

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices
Ismail Nejjar, Qin Wang, Olga Fink
Domain Adaptation Gram Matrix Domain Adaptation Benchmark

March 20, 2023

Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramian
Patricia Pauli, Ruigang Wang, Ian R. Manchester, Frank Allgöwer
Convolutional Neural Network Convolutional Layer Robustness Guarantee Optimization Layer Gram Matrix

February 11, 2023

Global Convergence Rate of Deep Equilibrium Models with General Activations
Lan V. Truong
Gradient Descent Activation Function Targeted Activation Penalty Deep Equilibrium Model Gram Matrix Quadratic Loss Global Convergence Rate

September 27, 2022

Efficient Noise Filtration of Images by Low-Rank Singular Vector Approximations of Geodesics' Gramian Matrix
Kelum Gajamannage, Yonggi Park, Mallikarjunaiah Muddamallappa, Sunil Mathur
Generalized Geodesic Gram Matrix Singular Vector

September 23, 2022

The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matrices
Wanlun Ma, Derui Wang, Ruoxi Sun, Minhui Xue, Sheng Wen, Yang Xiang
Backdoor Attack Backdoor Detection Gram Matrix Universal Backdoor

June 8, 2022

Fast Kernel Methods for Generic Lipschitz Losses via $p$-Sparsified Sketches
Tamim El Ahmad, Pierre Laforgue, Florence d'Alché-Buc
Kernel Method Effective Sketch Robust Regression Gram Matrix Lipschitz Loss Sketch Based Fast Kernel

March 4, 2022

Geodesic Gramian Denoising Applied to the Images Contaminated With Noise Sampled From Diverse Probability Distributions
Yonggi Park, Kelum Gajamannage, Alexey Sadovski
Industrial Disturbing Noise Denoising Process Gram Matrix Denoising Performance Image Purification Unknown Distribution

February 16, 2022

Using the left Gram matrix to cluster high dimensional data
Shahina Rahman, Valen E. Johnson, Suhasini Subba Rao
High Dimensional Data Clustering Algorithm Gram Matrix Microarray Data

February 9, 2022

Leverage Score Sampling for Tensor Product Matrices in Input Sparsity Time
David P. Woodruff, Amir Zandieh
Kronecker Product Tensor Product Gram Matrix Leverage Score Sampling Polynomial Kernel Input Sparsity

February 7, 2022

CECILIA: Comprehensive Secure Machine Learning Framework
Ali Burak Ünal, Nico Pfeifer, Mete Akgün
Private Inference Privacy Preserving Computation Gram Matrix Secure Machine Learning

December 21, 2021

Multi-Modality Distillation via Learning the teacher's modality-level Gram Matrix
Peng Liu
Knowledge Distillation Cross Modal Different Modality Good Teacher Gram Matrix Multi Modal Distillation

December 2, 2021

The Representation Jensen-R\'enyi Divergence
Jhoan Keider Hoyos Osorio, Oscar Skean, Austin J. Brockmeier, Luis Gonzalo Sanchez Giraldo
Kernel Hilbert Space Inverse Divergence Jensen Shannon Divergence Gram Matrix R\'enyi Divergence Divergence Measure