Randomized Hadamard Transforms

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 16

Aug 31, 2026cs.LG

RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search

By introducing RSLM (Rotated Scaled Lloyd-Max), a family of training-free vector quantization codecs compressing embeddings to 1--4 bits per dimension, we reduce memory cost and memory bandwidth of a typical large-scale Approximate Nearest Neighbor (ANN) search system, while reducing its complexity and keeping or improving recall across multiple benchmark datasets. State-of-the-art systems filter candidates using coarse partitions, approximately score them to narrow the set, and then rescore the best with higher precision representations (often >=8 bits per dimension). Our relativized codecs can bring this down to 2--4 bits per dimension. We use the properties of the ANN system to encode residual vectors instead of full vectors, both for the approximate scoring phase and the rescoring phase. Since Maximum Inner Product Search (MIPS) is very sensitive to vector norms, we correct the

L_2

norms of quantized vectors. Our major innovation is that we correct the

L_2

norm of the final reconstructed vector rather than just the residual. Our rescaling replaces more complicated schemes, such as Anisotropic loss. The residualization scheme gives us a more favorable quality vs size trade-off than generic quantization methods. Our high-performance implementation leverages a block-wise cascaded Fast Walsh-Hadamard Transform (FWHT) with linear-like complexity, AVX SIMD-optimized codebooks, and a steganographic encoding of scaling factors for perfect cache-line alignment.

Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato +2

Google Zurich, Switzerland · Technical University of Munich Munich, Germany · University of Copenhagen Copenhagen, Denmark +1

Residual Vector Quantization Approximate Nearest-Neighbor Search Vector Database Randomized Hadamard Transforms Behavioral Embeddings Affectcodec+1

Ask PDF

Jul 11, 2026cs.LG

Qubit-Efficient Quantum Search for Hyperdimensional Decomposition via Logarithmic Encoding

Hyperdimensional Computing (HDC) represents symbols using high-dimensional hypervectors of dimension

D

. In hypervector decomposition, the objective is to recover

F

constituent hypervectors, each drawn from a codebook of size

N

, from a bound target hypervector. This requires searching over

N^F

candidate tuples, making the task computationally prohibitive at scale. Recent quantum approach provides a quadratic search advantage, but typically rely on qubit-inefficient

O(D)

-qubit hypervector representations. We propose a qubit-efficient quantum framework for HDC decomposition that reduces the representation cost to

O(\log D)

. The framework introduces logarithmic hypervector and binding encodings, together with a reversible hypervector lookup operator for circuit-level manipulation of dense hypervectors. Combined with a modified Dürr-Høyer search procedure, the method preserves

O(\sqrt{N^F})

search complexity while substantially reducing qubit usage. Experimental results validate correct similarity computation, accurate decomposition in executable regimes, and significantly improved qubit scaling over baselines based on explicit

D

-qubit hypervector encodings, achieving up to

2{,}000\times

fewer qubits.

Sanggeon Yun, Hyunwoo Oh, Ryozo Masukawa +2

University of California, Irvine

Quantum Error Correction Quantum Circuits Randomized Hadamard Transforms Logarithmic Scaling

Ask PDF

Jun 22, 2026cs.LG

HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

We present HyperQuant (Hadamard, optimallY Packing, Entropy Rice-coding), a unified post-training quantization pipeline for the weights and the KV cache of large language and diffusion transformers. Across a suite of self-contained experiments (Table 1), HyperQuant outperforms the recent HIGGS scheme at every operating point from 3 to 5 bits per scalar (bps) on weights, and beats both TurboQuant and OCTOPUS on KV quantization down to 1.7 bps. Beyond the LLM setting, HyperQuant quantizes the 19B-parameter LTX-2 DiT video model with no observable per-frame artifacts. End-to-end on an H100 at 4 bps, HyperQuant compresses the linear weights ~3.9x and the KV cache ~3.79x at near-lossless quality. HyperQuant combines four known ideas into a single construction: (i) a per-tile Randomized Hadamard Transform that makes the per-coordinate distribution of weights and activations approximately Gaussian; (ii) quantization to a low-dimensional optimal lattice (E8, D4, A2, or Z); (iii) lossless bit-stripping and near-entropy-optimal variable-length Rice coding of the lattice indices; and (iv) bias-correction methods for the KV cache that keep the reconstruction unbiased under inner products, preserving attention semantics. We further integrate the pipeline with 8-bit and 4-bit Tensor-Core MMA paths (fp8-e4m3, int8, nvfp4, mxfp4), and find that int8 beats fp8 on the post-RHT lattice output. Project page: https://moonmath.ai/hyperquant/

Yuval Domb, Hadar Sackstein, Tomer Solberg

Post-Training Quantization Qantis Randomized Hadamard Transforms Diffusion Models

Ask PDF

Jun 12, 2026cs.CV

ShearFuse-UNet: Hadamard, DCT, and Shearlet Transform Fusion for Next-Day Wildfire Spread Prediction

We propose ShearFuse-UNet, a lightweight and computationally efficient deep learning model for next-day wildfire spread prediction from multi-modal satellite data. The model integrates three complementary transform-domain branches inside each encoder block of a U-Net backbone: a 2D Fast Walsh-Hadamard Transform (WHT) branch, a 2D Discrete Cosine Transform (DCT) branch, and a cone-adapted digital Shearlet residual branch. The WHT and DCT branches establish orthogonal latent spaces with learnable spectral scaling and fixed soft-thresholding, while the Shearlet branch provides anisotropic, multi-directional feature decomposition that explicitly encodes the elongated edge structures characteristic of fire fronts. A learned SpectralFusion gate adaptively combines the WHT and DCT responses, and the Shearlet reconstruction is added as a residual. This three-branch design bears a loose structural analogy to transformer self-attention: the WHT and DCT branches provide complementary spectral representations that are adaptively fused, while the Shearlet branch contributes directional content through a residual pathway. Unlike self-attention, the proposed design relies on fixed mathematical transforms rather than learned projection operators, reducing parameter count and computational cost. Evaluated on the WildfireSpreadTS dataset, ShearFuse-UNet achieves an F1 score of 0.596 with only 267k parameters, outperforming a ResNet18-based U-Net (14M parameters, F1 = 0.589) and demonstrating a highly favorable accuracy-efficiency trade-off. Results on the Google Next-Day Wildfire Spread dataset further validate these findings across a different benchmark.

Ene Meco, Yingyi Luo, Emadeldeen Hamdan +2

Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL 60607 USA · US Forest Service, Pacific Wildland Fire Science Laboratory, Seattle, WA 98103 USA

Wildfire Prediction U-Net Spatio-Temporal Transformers Fourier Randomized Hadamard Transforms

Ask PDF

Jun 8, 2026cs.CV

Adversarial Attack and Disturbance Detection by Hadamard-Coded Output Representations for Object Detection and Semantic Segmentation

Conventional one-hot encodings often yield poorly calibrated models, being overconfident under attack, and letting entropy-based detection algorithms fail. Previous image classification works have demonstrated that Hadamard-coded output representations can improve adversarial robustness. However, attempts to integrate Hadamard codes into semantic segmentation fall far behind state-of-the-art models in mean intersection-over-union performance. Regarding object detection, such output encodings have not yet been investigated at all. Further, no prior art addressed intrinsic codeword inconsistencies or actually exploited intrinsic codeword redundancy. Accordingly, we first derive a novel decoding procedure for Hadamard codewords towards optimal class-wise probabilities, solving the underlying optimization problem by using the projection onto the probability simplex. Second, our optimization delivers a measure of prediction inconsistency. Third, we are the first to show how to exploit these inconsistencies for adversarial attack and disturbance detection. Fourth, we introduce HadamardNet, a framework employing Hadamard codes as output representations for semantic segmentation and object detection models and tasks. We conduct a comprehensive evaluation both on disturbances and adversarial attacks, achieving state-of-the-art perturbation detection performance for both tasks in only a single detection pass, while delivering equivalent or close-by reference performance on clean data. Code is available at https://github.com/ifnspaml/HadamardPerturbationDetection.

Lucas Görnhardt, Timo Bartels, Niklas Schwarz +1

Technische Universität Braunschweig, Germany

Semantic Segmentation Object Detection Open-Vocabulary Object Detection Randomized Hadamard Transforms

Ask PDF

May 31, 2026cs.LG

Riemannian Optimization for Hadamard Products of Low-Rank Matrices

The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.

Pratik Jawanpuria, Ankish Chandresh, Bamdev Mishra

Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay, India · Microsoft India

Riemannian Optimization Low-Rank Structure Riemannian Manifolds Randomized Hadamard Transforms Matrix Matrix Multiplication

Ask PDF

May 28, 2026cs.LG

HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization

Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activation outliers and anisotropic weight curvature. Existing incoherence-based PTQ methods mitigate this issue with fixed randomized Hadamard transforms (RHTs), which improve quantization robustness but cannot adapt the rotated basis to the layer, calibration distribution, or quantizer. We introduce HARP (Hadamard-preconditioned Adaptive Rotation Processor), a learnable structured two-sided orthogonal processor that replaces fixed Hadamard mixing while preserving exact full-precision equivalence. HARP represents each rotation as a product of sparse butterfly-like block-orthogonal stages, supports non-power-of-two dimensions through Mixed-Radix schedules, and initializes to the RHT processor up to a fixed permutation. Fitted only on calibration data, HARP adapts the quantization basis to each layer and backend. Across 2--4-bit settings on Llama models from 1B to 70B, HARP consistently improves perplexity and yields its clearest zero-shot gains at 2 bits; a 2-bit Qwen3-8B experiment shows the same transfer beyond the Llama family. HARP also preserves deployment efficiency: on Llama 2 7B at 2 bits, it reaches 128 tok/s, retaining 90% of RHT throughput (142 tok/s) and running approximately

2.1\times

faster than FP16 (61 tok/s).

Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov

1MIRAI · 2BRAIn Lab · 3Innopolis University

Post-Training Quantization Large Language Model Quantization Randomized Hadamard Transforms

Ask PDF

May 28, 2026cs.LG

How Much Is a Dataset Worth? Scaling Laws, the Vendi Score, and Matrix Spectral Functions

Neural scaling laws appraise data through dataset size, while the Vendi Score uses quantum entropy to measure dataset value. We show both that common neural-scaling-law objectives and the Vendi Score are submodular. We further show that the Vendi Score is a special case of a broader class of submodular objectives that we call matrix spectral functions. This also includes determinantal (DPP) objectives, as well as many others. We also introduce weakly matrix monotone functions and show how they lead to weakly submodular matrix spectral functions, yielding a broad family of practical objectives for data appraisal. We develop secular-equation-based updates that avoid repeated eigendecompositions during greedy optimization, reducing marginal-gain evaluation for

m

-dimensional embeddings by an

O(m)

factor relative to oracle queries. This yields an average empirical speedup of about 35,000x, making direct optimization of the Vendi Score feasible on ImageNet-1K-scale datasets. Thus enabled, we compare how well several objectives predict the value of training subsets for held-out test performance under fixed-size, class-balanced, and fixed training-budget regimes, including the Vendi Score, DPPs, facility location, and three new matrix spectral variants. Across multiple datasets, facility location performs the best. Direct optimization also reveals that, while the Vendi Score is predictive over moderate score ranges, pushing the objective to higher values can make it a poor downstream performance proxy. We also find that uniformly at random fixed-size subsets, both unconstrained and class-balanced, are remarkably concentrated in both appraisal scores and held-out performance. Finally, we show that size, class balance, and training budget do not alone determine data value: even when controlling for these factors, performance ranges smoothly from good to bad.

Jeff A. Bilmes, Gantavya Bhatt, Arnav M. Das

Department of Electrical & Computer Engineering 2Paul G. Allen School of Computer Science & Engineering University of Washington, Seattle Seattle, WA 98195

Benchmark Datasets Destination-Only Scoring Objective Functions Scaling Laws Imagenet Randomized Hadamard Transforms

Ask PDF

May 27, 2026math.OC

Manifold-based Algorithms for the Hadamard Decomposition

Given a matrix

X

, and two ranks

r_1

and

r_2

, the Hadamard decomposition (HD) looks for two low-rank matrices,

X_1

of rank

r_1

and

X_2

of rank

r_2

, both of the same size as

X

, such that

X\approx X_1\circ X_2

, where

\circ

is the Hadamard (element-wise) product. In most cases, HD is more expressive than standard low-rank approximations such as the truncated singular value decomposition (TSVD), as it can represent higher-rank matrices with the same number of parameters; this is because the rank of

X_1 \circ X_2

is generically equal to

r_1 r_2

. In this paper, we first present some theoretical insights for HD, in particular a useful reformulation

X\approx WH^\top

where

W

and

H

have

r_1 r_2

columns and belong to certain manifolds. These allow us to develop three new algorithms for computing HD. The first one uses the representation

X\approx X_1\circ X_2

and relies on the Manopt toolbox. The other two rely on the reformulation

X\approx WH^\top

: one is a block projected gradient method, and the other is a manifold-based gradient descent algorithm that does not require projection onto the feasible set. The last two algorithms are particularly effective for handling large sparse data. We also propose new initializations that allow us to improve the accuracy of the HD. We compare our algorithms and initialization strategies with the TSVD and with the state of the art. Numerical results show that the new methods are efficient and competitive on both synthetic and real data.

Nicolas Gillis, Subhayan Saha, Stefano Sicilia +1

Department of Mathematics and Operational Research, University of Mons, Mons, Belgium. · SS is a member of the Gruppo Nazionale Calcolo Scientifico-Istituto Nazionale di Alta Matematica (GNCS-INdAM).

Randomized Hadamard Transforms Singular Value Decomposition Low-Rank Structure Decomposition

Ask PDF

May 25, 2026cs.LG

QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling

Scalar post-training quantizers discard pairwise coordinate structure within weight rows. We introduce QAM-W (Quadrature Amplitude Modulation for Weights), a codec that recovers this structure: each row is L2-normalized, block-Hadamard rotated, paired into 2D coordinates, and quantized against a single Lloyd-Max codebook trained on the unit circular Gaussian, with activation-aware per-channel scaling. In a cross-model study spanning five LLMs from four families (1.1B--13B parameters) and eight quantized configurations, the activation-aware variant at

\approx 5.5

bpw stays within

\pm 0.4\%

of BF16 WikiText-2 perplexity on every model, matching the SmoothQuant W8A8 quality envelope at

32\%

fewer weight bits. Joint 2D coding outperforms polar (amplitude

\times

phase) coding by 2--15pp $Δ$ PPL at equal bitrate, and paired KL against BF16 tracks $Δ$ PPL% at Spearman $ρ= 0.99$ across 37 (method, model) rows, consistent with a monotone composite bound from codec distortion to KL divergence. A 3.5bpw variant is competitive on quantization-tolerant architectures. At strict 4~~bpw, the rotated-codebook frontier method QTIP outperforms QAM-W; the contribution is the quality-preserving 5--6~~bpw band.

Preetam Sharma, Kacper Dobek

Independent Research · Institute of Computing Science Poznan University of Technology, Poznan, Poland

Model Weights Randomized Hadamard Transforms Distortion

Ask PDF

May 13, 2026cs.LG

Provable Quantization with Randomized Hadamard Transform

Vector quantization via random projection followed by scalar quantization is a fundamental primitive in machine learning, with applications ranging from similarity search to federated learning and KV cache compression. While dense random rotations yield clean theoretical guarantees, they require

Θ(d^2)

time. The randomized Hadamard transform

HD

reduces this cost to

O(d \log d)

, but its discrete structure complicates analysis and leads to weaker or purely empirical compression guarantees. In this work, we study a variant of this approach: dithered quantization with a single randomized Hadamard transform. Specifically, the quantizer applies

HD

to the input vector and subtracts a random scalar offset before quantizing, injecting additional randomness at negligible cost. We prove that this approach is unbiased and provides mean squared error bounds that asymptotically match those achievable with truly random rotation matrices. In particular, we prove that a dithered version of TurboQuant achieves mean squared error

\bigl(π\sqrt{3}/2 + o(1)\bigr) \cdot 4^{-b}

b

bits per coordinate, where the

o(1)

term vanishes uniformly over all unit vectors and all dimensions as the number of quantization levels grows.

Ying Feng, Piotr Indyk, Michael Kapralov +2

Randomized Hadamard Transforms Residual Vector Quantization Random Key-Value Cache Compression

Ask PDF

May 7, 2026cs.LG

Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Proven

Uniform random rotations (URRs) are a common preprocessing step in modern quantization approaches used for gradient compression, inference acceleration, KV-cache compression, model weight quantization, and approximate nearest-neighbor search in vector databases. In practice, URRs are often replaced by randomized Hadamard transforms (RHTs), which preserve orthogonality while admitting fast implementations. The remaining issue is the performance for worst-case inputs. With a URR, each coordinate is individually distributed as a shifted beta distribution, which converges to a Gaussian distribution in high dimensions. Generally, one RHT is not suitable in the worst case, as individual coordinates can be far from these distributions. We show that after composing two RHTs on any

d

-sized input vector, the marginal distribution of every fixed coordinate of the normalized rotated vector is within

O(d^{-1/2})

of a standard Gaussian both in Kolmogorov distance and in

1

-Wasserstein distance. We then plug these bounds into the analyses of modern compression schemes, namely DRIVE and QUIC-FL, and show that two RHTs achieve performance that asymptotically matches URRs. However, we show that two RHTs may not be sufficient for Vector Quantization (VQ), which often requires weak correlation across fixed-size blocks of coordinates (as opposed to only marginal distribution convergence for single coordinates). We prove that a composition of three RHTs leads to decaying coordinate covariance. This ensures that any fixed, bounded, multi-dimensional VQ codebook optimized for URRs has the same expected error when using three RHTs, up to an additive term that vanishes with the dimension. Finally, because practical inputs are rarely adversarial, we propose a linear-time

{O}(d)

check on the input's moments to dynamically adapt the number of RHTs used at runtime to improve performance.

Ran Ben-Basat, William Kuszmaul, Michael Mitzenmacher +2

UCL and Broadcom · Carnegie Mellon University · Harvard University +2

Randomized Hadamard Transforms Residual Vector Quantization Quantized Random

Ask PDF

May 2, 2026cs.LG

The Banach-Butterfly Invariant: Influence-Adaptive Walsh Geometry for Ternary Polynomial Threshold Functions

We introduce the Banach-Butterfly Invariant (BBT), an influence-adaptive Banach geometry on the Walsh-Hadamard butterfly factorization. For a Boolean function

f:\{-1,+1\}^n\to\{-1,+1\}

with coordinate influences

\mathrm{Inf}_\ell(f)

, BBT assigns exponent

p_\ell = 1+\mathrm{Inf}_\ell(f)

to butterfly layer

\ell

, yielding the contraction invariant

μ(f)=\prod_\ell 2^{-\mathrm{Inf}_\ell/(1+\mathrm{Inf}_\ell)}

. We prove a Jensen lower bound

\log_2μ(f) \ge -I(f)/(1+I(f)/n)

and that

μ

is strictly Schur-convex in the influence vector (modulo permutation), giving scaling classes

μ\sim 2^{-n/2}

(parity),

2^{-Θ(\sqrt{n})}

(majority),

2^{-1/2}

(dictators).

\log_2μ

is rational but not polynomial in the Fourier coefficients while

μ

is algebraic, and

μ

separates functions with identical total influence (122 pairs at

n=3

). Using the certified

n \le 4

ternary Walsh-threshold universe from a companion synthesis manuscript as a finite testbed, we compute exact MILP minimum-support certificates for all 65,536 Boolean functions at

n=4

(mean 6.42, max 9, all-odd by a parity argument) and on 10,000 of the 616,126 NPN-canonical representatives we enumerate at

n=5

(matching OEIS A000370). Conditional Spearman

ρ(μ,|\mathrm{supp}|)

at fixed total influence is

+0.571

in the largest stratum at

n=4

but reverses to

-0.38

n=5

under both function-uniform and NPN-canonical sampling:

μ

is a valid Schur-convex concentration invariant, not a universal monotone predictor of minimum support across

n

. A companion application paper validates a real-valued WHT activation-energy proxy inspired by this theory on five pretrained LLMs at W2A16, cutting wikitext-2 perplexity by 15-58% versus vanilla auto-round; the transfer from Boolean theory to the real-valued proxy is qualitative, not formal.

Gorgi Pavlov

Lehigh University · Johnson and Johnson

Polynomials Decision Thresholds Spectral Norm Randomized Hadamard Transforms Theory

Ask PDF

Apr 25, 2026cs.LG

Approximating Uniform Random Rotations by Two-Block Structured Hadamard Rotations in High Dimensions

Uniform random rotations are a useful primitive in applications such as fast Johnson-Lindenstrauss embeddings, kernel approximation, communication-efficient learning, and recent AI compression pipelines, but they are computationally expensive to generate and apply in high dimensions. A common practical replacement is repeated structured random rotations built from Walsh-Hadamard transforms and random sign diagonals. Applying the structured random rotation twice has been shown empirically to be useful, but the supporting theory is still limited. In this paper we study the approximation quality achieved when using this two-block structured Hadamard rotation. Our results are both positive and negative. On the positive side, we prove that every fixed coordinate of the two-block transform converges uniformly, over all inputs, to the corresponding coordinate of a uniformly rotated vector, with an explicit Kolmogorov-distance bound of order

d^{-1/5}

. On the negative side, we prove an explicit lower bound on the Wasserstein distance between the full vector distributions, showing that the two-block transform is not a globally accurate surrogate for a uniform random rotation in the worst case. For the extremal input used in the lower bound, we also prove a matching asymptotic upper bound, showing that the lower-bound scale is sharp for that input. Taken together, the results identify a clear separation between one-dimensional marginal behavior, where approximation improves with dimension, and full high-dimensional geometry, where a nonvanishing discrepancy remains. This provides a partial theoretical explanation for the empirical success of structured Hadamard rotations in some algorithms, while also clarifying the limitations of treating them as drop-in replacements for true uniform random rotations.

Tomer Zilca, Gal Mendelson

Faculty of Data and Decision Sciences, Technion – Israel Institute of Technology, Haifa, Israel · Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC, USA

Randomized Hadamard Transforms Random Low-Rank Structure

Ask PDF

Apr 21, 2026cs.LG

SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving

KV-cache memory is a major bottleneck in real-world LLM serving, where systems must simultaneously support latency-sensitive small-batch requests and high-throughput concurrent workloads. Although many KV-cache compression methods improve offline accuracy or compression ratio, they often violate practical serving constraints such as paged memory layouts, regular memory access, and fused attention execution, limiting their effectiveness in deployment. In this work, we identify the minimal set of 4-bit KV-cache quantization methods that remain viable under these constraints. Our central finding is that a simple design--token-wise INT4 quantization with block-diagonal Hadamard rotation--consistently achieves the best accuracy-efficiency trade-off. Across multiple models and benchmarks, this approach recovers nearly all of the accuracy lost by naive INT4, while more complex methods such as vector quantization and Hessian-aware quantization provide only marginal additional gains once serving compatibility is taken into account. To make this practical, we implement a fused rotation-quantization kernel that integrates directly into paged KV-cache layouts and introduces zero measurable end-to-end overhead, matching plain INT4 throughput across concurrency levels. Our results show that effective KV-cache compression is fundamentally a systems co-design problem: under real serving constraints, lightweight block-diagonal Hadamard rotation is a viable method that delivers near-lossless accuracy without sacrificing serving efficiency.

Jinda Jia, Jisen Li, Zhongzhu Zhou +8

Together AI

Key-Value Cache Quantization Key-Value Cache Compression Large Language Model Serving Randomized Hadamard Transforms Structural Constraints

Ask PDF

Mar 2, 2026cs.CV

Toward Generalizable Deep Learning Based Peatland Fire Detection via Walsh Hadamard Transform and Domain Adaptation

Machine learning-based wildfire detection has advanced significantly using deep learning models trained on large wildfire image and video datasets. However, peatland fires exhibit distinct characteristics, including smoldering combustion, low flame intensity, persistent smoke, and subsurface burning, limiting the effectiveness of conventional wildfire detectors. To address these challenges, we propose an efficient deep learning framework for peatland fire detection based on a Walsh--Hadamard Transform enhanced ResNet-50 (WHT-ResNet-50), which improves feature representation while reducing model complexity. To enable efficient deployment, the training-time architecture is structurally reparameterized into an equivalent inference model without sacrificing detection performance. Furthermore, the proposed framework leverages wildfire-to-peatland domain adaptation through transfer learning and introduces a mixed-domain training strategy that produces a unified detector capable of recognizing both wildfire and peatland fire events. Experimental results demonstrate that transfer learning substantially improves peatland fire detection under limited-data conditions, while the proposed WHT-ResNet-50 achieves higher accuracy and F1-score than conventional architectures with fewer parameters. The structurally reparameterized model further reduces inference cost while preserving detection accuracy. Video-based evaluation demonstrates robust performance with low false alarm rates, achieving a 100% event detection rate across all positive test videos. Overall, the proposed framework provides an accurate, efficient, and practical solution for early peatland fire detection.

Emadeldeen Hamdan, Ahmad Faiz Tharima, Mohd Zahirasri Mohd Tohir +4

Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL, USA · Fire and Rescue Department of Malaysia, Malaysia · Dept. of Chemical and Environmental Engineering, Faculty of Engineering, Universiti Putra Malaysia, Malaysia. +2

Wildfire Detection Resnet-18 Deep Learning Framework Domain Adaptation Randomized Hadamard Transforms

Ask PDF

Momentum

Related topics

Latest papers 16