The move of LLM inference to edge AI accelerators introduces new physical vulnerabilities. During execution, model parameters and intermediate inference states are repeatedly loaded into and processed on the chip, making them suscep- tible to physical side-channel attacks. In this work, by deploying laser voltage imaging, we show that one can extract LLM assets during inference, namely embeddings, attention, and quantized MLP weights, activations, and other inference states, from localized memories and compute subcircuits. To validate our claims, we perform an attack on an FPGA-based LLM accelerator. Since such accelerators reuse the same buffers and compute subcircuits across addresses, tiles, modules, and layers, reading asset values comes down to probing different memories during inference. We demonstrate full recovery of the targeted values; however, we also establish a methodology to recover asset values even if some weights or bits remain unread. We further derive lower bounds that relate imaging effort to asset dimensions and show that even direct recovery scales linearly with the size of the targeted asset
Figures & tables
Figure 1: Model and inference-state assets are repeatedly staged in the same classes of memory and compute structures. Direct bit-level reads provide exact anchors, while missing numerical entries or individual bits can be completed using linear input-output relations or recovered downstream states. Here, L denotes the number of transformer layers, dmodel the hidden dimension, and ∣V∣ the vocabulary size; Θ(⋅) denotes asymptotic bound. X is the module input, W the model weights, Y the module output, f(⋅) the known downstream computation, and Z a later state derived from Y .
Figure 2: Asset flow and on-chip storage during FPGA LLM inference. Model parameters and inference-state assets are repeatedly read from and written to reused memory and compute structures as inference proceeds across layers.
Design
Flip-flops
BRAM
URAM
Security-relevant implementation characteristic
LlamaF [ 43 , 25 ]
≈171 K
223
–
TinyLlama-scale embedded inference with quantized layer weights, scale factors, BRAM-cached activations, and pipelined matrix-vector computation.
Pushing up to the Limit [ 26 , 25 ]
≈105 K
≈37
10
LLaMA2-7B W4 inference on KV260 with a 4 GB external-memory budget, streamed weights, on-chip hidden state, and partial KV-cache storage.
Hummingbird [ 25 ]
25,422
59
18
LLaMA3-8B embedded inference with a 512-bit weight bus, 4-bit model weights, 8-bit KV cache, and time-multiplexed on-chip KV-cache buffering.
FlightLLM [ 44 ]
943K
1,252
792
U280 implementation with a large HBM-backed model, on-chip decode activations, and a buffer subsystem using 816 BRAMs and all 792 URAMs.
Table 1: Published FPGA LLM resource scales relevant to EOFM image planning.
Figure 3: EOFM results for FFs representing the input and output registers of the matrix multiplier. Green circles indicate input bits, while orange circles highlight the corresponding output bits for each evaluated input byte.
Figure 4: EOFM results for BRAM outputs. The left panel provides an overview of the selected BRAM region and the blue box denotes the field of view used for high-magnification EOFM. The remaining panels show the EOFM maps for representative data patterns.
Figure 5: EOFM results for a 16-bit BRAM configured with internal output registers. Orange circles denote active bit locations, while bounding boxes mark the physical output register logic. Purple boxes enclose 2-bit register pairs, and green boxes delineate single-bit edge registers. The 16-bit output is arranged across two 8-bit rows.
Figure 6: Representative 50% observation availability. The complete orange output arrays for 0xAA and 0x66 are artificially withheld in post-processing. All four panels originate from the EOFM captures in Figure 3 .
Figure 7: Joint recovery when both W and Y are incomplete. Eight bits of W and eight bits of each downstream state are treated as unreadable. Applying the EOFM observations in the order 0xFF , 0x11 , 0x66 , and 0xAA reduces the candidate set from 256 to 4 , 4 , 2 , and finally one candidate.
Figure 8: Four missing input bits from the representation of 0x11 are introduced in post-processing. The complete orange downstream output remains available and uniquely selects the original input completion.
Figure 9: Photon-emission results for sparsely placed FFs forming the matrix-multiplier input registers. The top-left panel shows the Vivado floorplan, while the remaining panels show representative data values. Blue boxes mark FF locations and orange circles mark active logic ‘1’ states.
On-device Vision-Language Models (VLMs) promise data privacy via local execution. However, we show that the architectural shift toward Dynamic High-Resolution preprocessing (e.g., AnyRes) introduces an inherent algorithmic side-channel. Unlike static models, dynamic preprocessing decomposes images into a variable number of patches based on their aspect ratio, creating workload-dependent inputs. We demonstrate a dual-layer attack framework against local VLMs. In Tier 1, an unprivileged attacker can exploit significant execution-time variations using standard unprivileged OS metrics to reliably fingerprint the input's geometry. In Tier 2, by profiling Last-Level Cache (LLC) contention, the attacker can resolve semantic ambiguity within identical geometries, distinguishing between visually dense (e.g., medical X-rays) and sparse (e.g., text documents) content. By evaluating state-of-the-art models such as LLaVA-NeXT and Qwen2-VL, we show that combining these signals enables reliable inference of privacy-sensitive contexts. Finally, we analyze the security engineering trade-offs of mitigating this vulnerability, reveal substantial performance overhead with constant-work padding, and propose practical design recommendations for secure Edge AI deployments.
Eyal Hadad, Mordechai Guri
Ben-Gurion University of the Negev, Israel Faculty of Computer and Information Science
Large language models (LLMs) achieve high inference efficiency using optimized GPU kernels generated by just-in-time (JIT) compilation frameworks such as Triton. Triton compiles GPU kernels through an MLIR/LLVM-based pipeline. We investigate whether a Rowhammer-induced bit-flip in the CPU's DRAM can degrade GPU-based LLM inference performance while preserving output correctness. We present LLVMhammer, the first Rowhammer attack that causes LLM inference slowdown by inducing a bit-flip in Triton's compilation pipeline. A bit-flip in a performance-critical optimization pass causes Triton to generate a functionally identical but suboptimal GPU kernel. Since Triton caches the generated kernel on disk, a single successful bit-flip causes persistent slowdown across subsequent inference executions. Triton's target-independent stages are shared across GPU vendors, which makes our attack GPU vendor-agnostic. To identify exploitable bit-flip locations without manual inspection, we build Karnage, a tool that isolates useful bit-flip locations in Triton's target-independent compilation pipeline. We evaluate LLVMHammer across a diverse set of LLMs, including Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Deepseek-llm-7B-chat, across NVIDIA RTX 4090 and AMD RX 7900 XTX GPUs. LLVMhammer increases inference latency of an LLM inference server running on a single GPU by 4-16%, which, in turn, can translate into additional annual operating costs at the scale of multi-GPU deployments. We further show that LLVMhammer can also corrupt LLM outputs across different GPU vendors, reducing BERTScore to as low as 25%, illustrating the broader security implications of bit-flips in Triton's compilation pipeline.
Mrityunjay Shukla, Shubham Roy, Biswabandan Panda
Indian Institute of Technology Bombay, Mumbai, India
Large language models (LLMs) are increasingly deployed on edge devices under strict computation and quantization constraints, yet their security implications remain unclear. We study query-based knowledge extraction from quantized edge-deployed LLMs under realistic query budgets and show that, although quantization introduces noise, it does not remove the underlying semantic knowledge, allowing substantial behavioral recovery through carefully designed queries. To systematically analyze this risk, we propose CLIQ (Clustered Instruction Querying), a structured query construction framework that improves semantic coverage while reducing redundancy. Experiments on quantized Qwen models (INT8/INT4) demonstrate that CLIQ consistently outperforms original queries across BERTScore, BLEU, and ROUGE, enabling more efficient extraction under limited budgets. These results indicate that quantization alone does not provide effective protection against query-based extraction, highlighting a previously underexplored security risk in edge-deployed LLMs.
Ao Ding, Hongzong Li, Zi Liang +5
China University of Geoscience Beijing · Hong Kong University of Science and Technology · Hong Kong Polytechnic University +4