AIR-LLM: Broadcasting AI Weights over Radio for Memory-Free Edge LLM Inference via RF Computing

Zhihui Gao; Tingjun Chen; Dirk Englund

cs.ITSep 30, 2026

AIR-LLM: Broadcasting AI Weights over Radio for Memory-Free Edge LLM Inference via RF Computing

Authors: Zhihui Gao, Tingjun Chen, Dirk Englund

Organizations: Massachusetts Institute of Technology Cambridge, MA, USA · Duke University Durham, NC, USA

Abstract

Next-generation large language models (LLMs) are expanding from the cloud to ubiquitous edge devices. However, edge devices typically either lack the memory to store increasingly large LLM weights or, even with enough memory, spend unaffordable energy on loading the weights. This raises our question: can an edge device run an LLM without storing or loading its weights, but receive them over the air and consume them on the fly? Inspired by wireless broadcasting, we present AIR-LLM, an LLM inference architecture for edge devices, which is composed of: (i) a central radio (e.g., 5G base stations) that broadcasts the LLM weights into the air, and (ii) the edge user that receives the weights and completes the general matrix-vector multiplication (GEMV) of LLM inference directly in the radio frequency (RF) domain using RF mixers. To further shorten the airtime, AIR-LLM exploits MIMO spatial multiplexing and proposes an energy-efficient precoder-postcoder pair on the edge to calibrate its own wireless channel. Since the central radio stays user-unaware, AIR-LLM is user-scalable so that one broadcast serves unlimited users within its coverage. We implement AIR-LLM on the NVIDIA Sionna ray-traced channels of two real-world urban scenes and the profiling of a real RF mixer. With a WikiText-2 perplexity degradation of 4.0% on LLaMA-3.1-8B, AIR-LLM saves the energy by 157.7x/40.4x against the FP16 and weight-only quantization baselines; with 20 users, its airtime is 104.1x/26.0x shorter, respectively.

Figures & tables

Figure 1 . System overview of AIR-LLM : a central radio frequency-encodes the LLM weights onto an OFDM waveform and broadcasts it over the air to serve any edge user within the coverage; the edge user drives the LO with its private activations, and the RF mixer’s output directly carries the GEMV outputs. The LLM inference is completed once the weight waveform arrives, without ever storing it in memory.

Computing Paradigm	Memory- free	Energy- efficient	Prompt- private	Wireless (untethered)	User- scalable	LLM-scale GEMV
Cloud computing ( Kwon et al., 2023 ; Stojkovic et al., 2025 ; Kang et al., 2017 )	✓	✓	✗	✓	✗	✓
Edge computing ( Frantar et al., 2023 ; Lin et al., 2024 ; Song et al., 2024 ; Alizadeh et al., 2024 )	✗	✗	✓	✓	✓	✓
In-memory computing ( Shafiee et al., 2016 ; Wan et al., 2022 ; Le Gallo et al., 2023 )	✓	✓	✓	✗	✓	✗
Photonic computing ( Shen et al., 2017 ; Zhong et al., 2023 ; Sludds et al., 2022 )	✗	✓	✓	✗	✓	✗
Over-the-air computing ( Nazer and Gastpar, 2007 ; Zhu et al., 2020 ; Reus-Muns et al., 2023 )	✗	✗	✓	✓	✗	✗
AIR-LLM	✓	✓	✓	✓	✓	✓

Table 1 . The comparison of AIR-LLM and existing edge computing paradigms.

Figure 2 . The overview of AIR-LLM ’s RF computing engine, which completes $A$ IPs of the GEMV per round (or $2A$ if real-valued). The central radio broadcasts the weights of these IPs over its $A$ antennas, without user-specific precoding. Each edge user then feeds the $A$ received weight waveforms, together with its own GEMV input, into $C$ calibration units, whose precoders and postcoders are derived from that user’s own channel by an SVD-based initialization and an ALS-based fine-tuning. Summing the outputs of the $C$ units gives the GEMV output of this round.

Figure 3 . The detailed calibration unit architecture and its handling of an inner product between the received weight vector

\mathbf{w}^{\prime}_{a}

and the precoded input vector

\mathbf{x}^{\prime}

Panel of Figure 4 — Figure 4 . The two Sionna ray-tracing scenes in Paris (a) and Munich (b), and the testbed for RF mixer profiling.

Panel of Figure 5 — Figure 5 . In both Paris and Munich, the ENOB of a randomized GEMV over user PL (a) and broadcast power (b).

Panel of Figure 6 — Figure 6 . With $A=\left\{2^{1},\dots,2^{5}\right\}$ , the ENOB over $C$ or energy (a), and the minimum energy to achieve 2–4 ENOBs in Paris.

Panel of Figure 7 — Figure 7 . When scaling up the GEMV sizes, the ENOB (a) and the energy per FLOP breakdown of AIR-LLM (b).

Precision	Method	LLaMA-3				LLaMA-2		Mistral	Gemma-4
Precision	Method	3.2-1B	3.2-3B	3.1-8B	3.1-70B †	7B	13B	7B	E2B	E4B
FP16	-	9.77	7.82	6.24	2.81	5.47	4.88	5.25	8.24	7.07
W4A16	RTN	11.71	8.49	6.83	3.35	5.72	4.98	5.42	10.60	193.64
	GPTQ	10.63	8.78	6.65	3.37	5.62	4.99	5.38	8.95	11.51
	AWQ	10.95	8.29	6.66	3.27	5.60	4.97	5.37	9.57	11.72
ENOB 4	AWGN	12.28	8.75	7.11	4.30	40.94	5.06	16.31	9.51	11.60

Table 2 . Over the nine LLMs, the WikiText-2 PPL of AIR-LLM on the selected users of Paris and Munich vs. the FP16, WoQ, and analog computing baselines, at the sequence length of 2,048.

Figure 8 . The normalized energy per token of AIR-LLM vs. four baselines over the nine LLMs.

Figure 9 . The normalized airtime per token of AIR-LLM vs. two remote baselines over the nine LLMs.

Precision/ User	Sequence length
Precision/ User	256	512	1,024	2,048	4,096	8,192	16,384
FP16	10.37	8.18	6.96	6.24	5.85	5.62	5.48
Paris	11.05	8.57	7.26	6.49	6.08	5.84	5.70
Munich	12.99	9.88	8.30	7.37	6.87	6.58	6.43

Table 3 . In the prefilling phase, the PPL of AIR-LLM in Paris and Munich over different sequence lengths.

User	Central radio transmit power (dBm)
User	35	40	45	50	55	60
Paris	6.81	6.59	6.51	6.49	6.49	6.48
Munich	$3.2{\times}10^{3}$	270.42	15.76	7.37	6.95	6.79

Table 4 . The PPL of AIR-LLM in Paris and Munich over different central radio broadcasting power.

Ant # = 4 @ 50 dBm			Ant # = 8 @ 60 dBm
Unit #	Paris	Munich	Unit #	Paris	Munich
2	$2.2{\times}10^{5}$	$6.7{\times}10^{5}$	8	$4.3{\times}10^{4}$	$1.2{\times}10^{4}$
4	64.48	140.60	16	14.65	24.58
6	7.46	9.26	24	7.68	8.37
8	6.49	7.37	32	6.74	6.95
12	6.33	7.41	48	6.50	6.59

Table 5 . The PPL of AIR-LLM in Paris and Munich at

A=\left\{4,8\right\}

with a varying number of calibration units,

C

Freq.	BW	Power	User	$A={4}$		$A={8}$
(GHz)	(MHz)	(dBm)	User	$C={8}$	$C={16}$	$C={32}$	$C={64}$
0.915	25	30/40	Paris	6.73	6.35	6.32	6.32
0.915	25	30/40	Mun.	7.32	7.35	6.62	6.72
3.5	100	50/60	Paris	6.49	6.32	6.74	6.50
3.5	100	50/60	Mun.	7.37	7.46	6.95	6.59
28	400	80/90	Paris	7.07	6.35	8.59	6.78

Table 6 . The PPL of AIR-LLM at ISM (0.915 GHz) and mmWave (28 GHz) bands with different

(A,C)

combinations.

Panel of Figure 10 — Figure 10 . The broadcasting of LLaMA-3.1-8B to the 20 users with WikiText-2 PPL labeled in Paris (a) and Munich (b).

Panel of Figure 11 — Figure 11 . In both scenes, the PPL CDFs vs. those by FP/WoQ baselines (a), and the energy per token comparison (b).

Figure 12 . The cumulative airtime over

U=\left\{1\dots 20\right\}

by AIR-LLM and the FP16/WoQ baselines in Paris.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 19 — Figure A1 . What the two steps of OPT-Chan buy on the selected users at $A={4}$ and $C={8}$ : $\hat{\mathbf{y}}$ against $\mathbf{y}$ at four stages of the solve (a, b), and the ( 17 ) loss and its ENOB per ALS iteration (c, d).

Term	Energy per module	Energy per FLOP
Precoding	$\mathcal{O}\left(CDL_{x}\right)$	$\mathcal{O}\left(\dfrac{C}{ML_{w}}\right)$
IFFT	$\mathcal{O}\left(CDL\log_{2}(DL)\right)$	$\mathcal{O}\left(\dfrac{CL\log_{2}(DL)}{ML_{w}L_{x}}\right)$
DAC	$\mathcal{O}\left(\dfrac{CMDL}{A}\right)$	$\mathcal{O}\left(\dfrac{CL}{AL_{w}L_{x}}\right)$
ADC	$\mathcal{O}\left(CML\right)$	$\mathcal{O}\left(\dfrac{CL}{DL_{w}L_{x}}\right)$
FFT	$\mathcal{O}\left(CML\log_{2}L\right)$	$\mathcal{O}\left(\dfrac{CL\log_{2}L}{DL_{w}L_{x}}\right)$
Postcoding	$\mathcal{O}\left(CMAL\right)$	$\mathcal{O}\left(\dfrac{CAL}{DL_{w}L_{x}}\right)$

Appendix

Table A3 . The energy complexity of one module on AIR-LLM under BCO and the seven terms per FLOP.

Term	Energy per GEMV	Energy per FLOP
Precoding	$\mathcal{O}\left(CN\right)$	$\mathcal{O}\left(\dfrac{C}{M}\right)$
IFFT	$\mathcal{O}\left(CN\log_{2}N\right)$	$\mathcal{O}\left(\dfrac{C\log_{2}N}{M}\right)$
DAC	$\mathcal{O}\left(\dfrac{CNM}{A}\right)$	$\mathcal{O}\left(\dfrac{C}{A}\right)$
ADC	$\mathcal{O}\left(CM\right)$	$\mathcal{O}\left(\dfrac{C}{N}\right)$
FFT	0	0
Postcoding	$\mathcal{O}\left(CMA\right)$	$\mathcal{O}\left(\dfrac{CA}{N}\right)$

Appendix

Table A4 . The energy complexity of one GEMV on AIR-LLM and the same seven terms per FLOP, along the signal chain.

Term	Energy per module	Energy per useful FLOP
Precoding	$\mathcal{O}\left(CNI^{2}\right)$	$\mathcal{O}\left(\dfrac{C}{MK^{2}}\right)$
IFFT	$\mathcal{O}\left(CNI^{2}\log_{2}(NI)\right)$	$\mathcal{O}\left(\dfrac{C\log_{2}(NI)}{MK^{2}}\right)$
DAC	$\mathcal{O}\left(\dfrac{CNMI^{2}}{A}\right)$	$\mathcal{O}\left(\dfrac{C}{AK^{2}}\right)$
ADC	$\mathcal{O}\left(CMI^{2}\right)$	$\mathcal{O}\left(\dfrac{C}{NK^{2}}\right)$
FFT	$\mathcal{O}\left(CMI^{2}\log_{2}I\right)$	$\mathcal{O}\left(\dfrac{C\log_{2}I}{NK^{2}}\right)$
Postcoding	$\mathcal{O}\left(CMAI^{2}\right)$	$\mathcal{O}\left(\dfrac{CA}{NK^{2}}\right)$

Appendix

Table A5 . The energy complexity of one Conv2D layer on AIR-LLM and the same seven terms per useful FLOP, along the signal chain, assuming

I\gg K

Figure A2 . On the wired testbed of Fig. 4 (c), the best ENOB of the ZEM-4300+ mixer over its RF port power.

Panel of Figure 24 — Figure A3 . Path-loss distribution across the candidate-device pool of each ray-tracing scene.

Panel of Figure 25 — Figure A4 . Fig. 6 in Munich: the ENOB over energy (a), and the minimum energy to achieve 2–4 ENOBs (b).

Panel of Figure 26 — Figure A5 . Figs. 8 and 9 in Munich: the normalized energy (a) and airtime (b) per token over the nine LLMs.

Panel of Figure 27 — Figure A6 . The 20 users of Fig. 10 in Paris (a) and Munich (b), each labeled with its PL, GEMV ENOB and WikiText-2 PPL.

Device	PL	ENOB	PPL	Generated continuation
14	70.0	5.71	6.31	… its statue of the Marianne, a symbol of the French Republic. The square is located in the 16th arr
6	76.6	5.70	6.32	… its central fountain and the surrounding traffic circle. It is located in the 17th arrondissement, in the north
19	78.8	5.64	6.31	… its central fountain. The square is located in the 8th arrondissement, at the intersection of the Champs
15	82.5	5.59	6.32	… its market. It is located in the 18th arrondissement, in the district of Montmartre. It
9	85.4	5.56	6.34	… the Arc de Triomphe at its center. The name Place de l’Étoile (Star Square) comes
10	82.3	5.49	6.32	… the fountain in the center, which is the work of the sculptor Louis Derbré. The square is located at

Appendix

Table A6 . What the twenty devices of Paris in Fig. A6 generate from the shared prompt “The Place de l’Étoile is a square in Paris, France, best known for…”, which the FP16 model continues as “… the Arc de Triomphe, located at its center. The place is named for the star-shaped design of the round”. PL is in dB, ENOB in bits, and PPL is measured on WikiText-2;

\hookleftarrow

marks a line break inside a continuation.

Device	PL	ENOB	PPL	Generated continuation
7	68.6	5.78	6.31	… its annual Christmas market. The city is also known for its beer gardens, which are open year-round. The city is
19	75.4	5.11	6.35	… its annual Christmas market. It is the capital of the state of Bavaria and is the third largest city in Germany after
18	87.5	4.81	6.36	… its annual Oktoberfest, a celebration of all things beer. It is also the home of the world’s largest beer hall
17	91.7	4.77	6.67	… its annual Christmas market. The market is a tradition that dates back to the 14th century, and is one of
12	105.0	4.19	6.60	… its ability to produce beer. The city is the capital of the state of Bavaria. The city is the home of
6	85.0	4.03	6.53	… its beer and Oktoberfest. It is also the home of the famous Hofbrau brewery, which is the oldest brewery

Appendix

Table A7 . What the twenty devices of Munich in Fig. A6 generate from the shared prompt “Munich is a city in southern Germany, best known for…”, which the FP16 model continues as “… its beer and Oktoberfest, the world’s largest beer festival. The city is also home to a number of museums,”. PL is in dB, ENOB in bits, and PPL is measured on WikiText-2;

\hookleftarrow

marks a line break inside a continuation.

Precision	Method	PIQA	ARC-e	ARC-c	BoolQ	HellaSwag	WinoGrande	Avg.
FP16	-	81.01	81.10	53.50	81.96	78.89	73.56	75.00
W4A16	RTN	80.47	78.41	52.22	81.22	77.93	73.72	74.00
	GPTQ	80.58	78.75	51.45	80.95	78.07	72.85	73.77
	AWQ	80.74	80.18	52.99	81.19	78.29	73.64	74.51
ENOB 4	AWGN	52.07	27.48	23.72	47.25	27.90	49.49	37.98
	HyFlexPIM	80.79	80.51	53.16	80.92	78.37	73.72	74.58

Appendix

Table A8 . Over six commonsense-reasoning benchmarks, the zero-shot accuracy (%) of LLaMA-3.1-8B by AIR-LLM on the selected users of Paris and Munich vs. the FP16, WoQ, and analog computing baselines.

Edge device	Repetitions of the weight signal
Edge device	1	2	4	8	16	$\infty$
Paris	6.49	6.49	6.49	6.49	6.49	6.49
Munich	7.37	7.27	7.22	7.20	7.19	7.17

Appendix

Table A9 . The WikiText-2 PPL of LLaMA-3.1-8B when the weight signal is repeated 1 to 16 times and averaged, with

\infty

as the distortion floor.

Band	Edge device	Central radio transmit power (dBm)
Band	Edge device	10	15	20	25	30	35	40	45	50
915 MHz	Paris	9.21	7.56	7.02	6.81	6.72	6.70	6.69	6.69	6.69
915 MHz	Munich	$5.9{\times}10^{5}$	$3.7{\times}10^{3}$	28.14	9.15	7.28	6.74	6.54	6.47	6.44
		60	65	70	75	80	85	90	95	100
28 GHz	Paris	9.31	7.65	7.22	7.10	7.07	7.06	7.05	7.06	7.06
28 GHz	Munich	$3.9{\times}10^{5}$	$4.3{\times}10^{5}$	$2.9{\times}10^{5}$	$3.1{\times}10^{4}$	170.87	21.48	12.59	10.76	9.98

Appendix

Table A10 . Tab. 4 repeated at the ISM (0.915 GHz) and mmWave (28 GHz) bands, each read against its own regulatory ceiling.

Ant # = 4			Ant # = 8
Unit #	Paris	Munich	Unit #	Paris	Munich
915 MHz : 30 dBm at $A={4}$ and 40 dBm at $A={8}$
2	$3.3{\times}10^{5}$	$4.2{\times}10^{5}$	8	79.04	94.78
4	77.08	27.44	16	7.25	6.51
6	8.81	7.71	24	6.43	6.54
8	6.73	7.32	32	6.32	6.62

Appendix

Table A11 . Tab. 5 repeated at the ISM and mmWave bands, each at the transmit power where it saturates in Tab. A10 .

Figure A7 . Fig. 12 in Munich, against the same 3-bit baseline.

Explore similar work

May 12, 2026cs.IT

CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference

As large language models (LLMs) move from centralized clouds to mobile edge environments, efficient serving must balance latency, energy consumption, and accuracy under constrained device-edge resources. Query-level routing between lightweight on-device models and stronger edge models provides a flexible mechanism to navigate this trade-off. However, existing routers are designed for centralized cloud settings and optimize token-level costs, failing to capture the dynamic latency and energy overheads in wireless edge deployments. In this paper, we formulate mobile edge LLM routing as a deployment-constrained, cost-aware decision problem, and propose CR^2, a two-stage device-edge routing framework. CR^2 decouples a lightweight on-device margin gate from an edge-side utility selector for deferred queries. The margin gate operates on frozen query embeddings and a user-specified cost weight to predict whether local execution is utility-optimal relative to the best edge alternative under the target operating point. We further introduce a conformal risk control (CRC) calibration procedure that maps each operating point to an acceptance threshold, enabling explicit control of the marginal false-acceptance risk under the full-information utility reference. Experiments on the routing task show that CR^2 closely matches a full-information reference router using only device-side signals before deferral. Compared with strong query-level baselines, CR^2 consistently improves the deployable accuracy-cost Pareto frontier and reduces normalized deployment cost by up to 16.9% at matched accuracy.

Nan Xue, Shengkang Chen, Zhiyong Chen +4

Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China · Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518000, China · Future Network of Intelligent Institute (FNii), the Chinese University of Hong Kong (Shenzhen), Shenzhen 518172, China +1

Large Language Model Routing Edge Devices LLM Inference Optimization Cost-Aware Cloud Pareto Frontier

Ask PDF

Aug 6, 2026cs.NI

BALANCE: Hybrid Autoregressive-Speculative LLM Inference in Wireless Edge Networks

Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference by using a small language model (SLM) to generate multiple draft tokens for LLM verification, but incurs extra memory costs. Due to this latency-memory tradeoff, neither approach alone can efficiently serve users with heterogeneous demands under limited edge computing resources. To address this challenge, we propose a hybrid autoregressive-speculative inference (BALANCE) framework for edge LLM inference. In BALANCE, an edge server hosts both an SLM and an LLM, assigns each user to AD or SD, and performs the two modes simultaneously. To maximize the number of served users, we formulate a task throughput maximization problem to jointly determine user scheduling and computing resource allocation between AD and SD under user latency requirements and server memory constraints. Since the problem is NP-hard, we develop a polynomial-time algorithm that transforms the original problem into two sub-problems and obtains a sub-optimal solution with a constant approximation guarantee. Experiments demonstrate that BALANCE consistently outperforms conventional AD and SD and significantly improves task throughput.

Guanqiao Qu, Shuo Chen, Qian Chen +2

Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · Electrical and Electronic Engineering Department, Imperial College, London SW7 2BT, U.K. · Electrical and Electronic Engineering, and Computing Departments, Imperial College, London SW7 2BT, U.K.

LLM Inference Optimization Autoregressive Decoding Edge Balance

Ask PDF

Jul 14, 2026cs.CR

Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

On-device LLM inference faces a trilemma of response latency, limited hardware resources and user privacy. Full cloud inference delivers strong computing power but exposes user prompts and dialogue data, while standalone on-device inference is unfeasible for most consumer and embedded edge devices. This paper presents a privacy-centric edge-cloud collaborative LLM inference framework built on endpoint-authenticated KV cache. Local endpoints handle input preprocessing, embedding computation, adaptive feature optimization, KV cache authentication, speculative decoding and low-dimensional model head calculation, while the cloud conducts authenticated decoder inference, KV cache management, token verification and high-dimensional vocabulary projection. Endpoints fuse partial outputs, apply language-adaptive masking and sample target tokens. All transmitted data and truncated logits are quantized and AES-GCM encrypted for privacy, with core lightweight modules, draft parameters and cache access policies kept local to avoid leakage. The framework supports heterogeneous devices including CPU-only, GPU-equipped and embedded devices via optimized streaming, batching and quantized ONNX deployment. Evaluations demonstrate that the framework reduces per-token latency by up to 46.1% and downlink payloads by up to 67.4% over baseline split inference, retaining comparable performance to full cloud inference.

Yi Li, Chen Li, Jiexiong Liu

KunlunMeta

LLM Inference Optimization Intelligent Edge Time-To-First-Token Cloud Edge Privacy+1

Ask PDF