cs.ITSep 22, 2026

Logit-Aware MIMO AirComp for Distributed Mixture-of-Experts LLM Inference over Wireless Edge Networks

Authors: Lyutianyang Zhang, Yunjian Jia, Liu Cao, Dengke Wang, Jinke Ren, Shuguang Cui

Organizations: School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, China · School of Science and Engineering, the Shenzhen Future Network of Intelligence Institute, and the Guangdong Provincial Key Laboratory of Future Networks of Intelligence, the Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong 518172, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Distributed mixture-of-experts (MoE) inference is a promising architecture for deploying large language models (LLMs) at wireless edge networks because sparse experts can be placed across coordinated base stations (BSs), while the anchor node and user equipment (UE) can offload LLM inference tasks to BSs. The communication bottleneck is the MoE aggregation, where the anchor BS must recover a weighted sum of selected expert outputs before each decoding step. Over-the-air computation (AirComp) is well matched to this operation because the wireless multiple-access channel naturally superposes simultaneous transmissions. However, conventional AirComp minimizes communication distortion, whereas MoE aggregation errors have unequal impact on LLM outputs. We propose a logit-aware MIMO AirComp framework that estimates local logit sensitivity as block-level weights and jointly optimizes receive combiners and BS precoders under per-BS power constraints. We also develop an alternating algorithm that protects decoding decisions from aggregation perturbations. Using OpenCompass, we evaluate Qwen3-30B-A3B-Instruct-2507-FP8 on GSM8K and ARC-Challenge. At 30 dB aggregation SNR, perturbed Qwen3 retains 99.3% of clean GSM8K accuracy and 94.8% of clean ARC-Challenge accuracy. In a direct ARC-Challenge closed-loop audit over 295 examples, SW-AirComp achieves 43.73% accuracy at 30 dB and 26.78% at 20 dB, outperforming unweighted, matched-filter, and zero-forcing AirComp. In the wireless simulator, SW-AirComp reduces decision-relevant aggregation distortion; at 20 dB, its final weighted-sum mean-squared error is 53.5% lower than unweighted AirComp under the same channels, samples, and sensitivity weights. Logit-RMSE and logit-gap diagnostics further show that the gain comes from reducing output-logit perturbation and lowering the risk of top-token changes.

Figures & tables

Fig. 1: System architecture of distributed MoE inference.
Fig. 2: Construction of scalar AirComp weights from block-level PHY error and local next-token logit sensitivity.
BenchmarkItemPrompt excerpt and choicesGold answer
GSM8K#0Janet’s ducks lay 16 eggs per day. She eats three for breakfast and uses four for muffins. She sells the remainder at the farmers’ market for $2 each.18
GSM8K#1A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?3
ARC-c#0Juan and LaKeisha roll a few objects down a ramp. They want to see which object rolls the farthest. What should they do so they can repeat their investigation? Choices: A) Put the objects in groups; B) Change the height of the ramp; C) Choose different objects to roll; D) Record the details of the investigation.D
ARC-c#1High-pressure systems stop air from rising into the colder regions of the atmosphere where water can condense. What will most likely result if a high-pressure system remains in an area for a long period of time? Choices: A) fog; B) rain; C) drought; D) tornado.C
TABLE I: Examples of OpenCompass Subsets
Fig. 3: Qwen3 OpenCompass accuracy under controlled MoE aggregation perturbations.
ParameterValue
Number of BSsM=4M=4
Active BSs per aggregation∣Mb∣=2|\mathcal{M}_{b}|=2
Transmit antennas per BSNT∈{32,64,128}N_{\rm T}\in\{32,64,128\}
Receive antennasNR=2N_{\rm R}=2
System bandwidth2020 MHz
Maximum BS transmit powerPmtot=1P_{m}^{\rm tot}=1 W ( 3030 dBm)
TABLE II: Wireless Parameters
Fig. 4: ARC-Challenge Accuracy: AirComp Perturbations.
Fig. 5: Block–SNR WMSE heatmap.
Fig. 6: Sensitivity-Weighted MSE vs SNR for Antenna Scaling.
Fig. 7: Massive-MIMO aggregation study over 16 wireless aggregation blocks.
Fig. 8: Normalized WSMSE convergence.
Fig. 9: (a) reports each baseline’s weighted-logit RMSE normalized by SW-AirComp. (b) reports the percentage of aggregation blocks whose estimated logit perturbation exceeds the clean top-token logit gap; lower is better.
More in this topic

LLM Inference

Large language model inference studies how models generate outputs efficiently, reliably, and securely across hardware and workloads.

See all 285 papers

Explore similar work

CardsList
Sep 30, 2026

AIR-LLM: Broadcasting AI Weights over Radio for Memory-Free Edge LLM Inference via RF Computing

Next-generation large language models (LLMs) are expanding from the cloud to ubiquitous edge devices. However, edge devices typically either lack the memory to store increasingly large LLM weights or, even with enough memory, spend unaffordable energy on loading the weights. This raises our question: can an edge device run an LLM without storing or loading its weights, but receive them over the air and consume them on the fly? Inspired by wireless broadcasting, we present AIR-LLM, an LLM inference architecture for edge devices, which is composed of: (i) a central radio (e.g., 5G base stations) that broadcasts the LLM weights into the air, and (ii) the edge user that receives the weights and completes the general matrix-vector multiplication (GEMV) of LLM inference directly in the radio frequency (RF) domain using RF mixers. To further shorten the airtime, AIR-LLM exploits MIMO spatial multiplexing and proposes an energy-efficient precoder-postcoder pair on the edge to calibrate its own wireless channel. Since the central radio stays user-unaware, AIR-LLM is user-scalable so that one broadcast serves unlimited users within its coverage. We implement AIR-LLM on the NVIDIA Sionna ray-traced channels of two real-world urban scenes and the profiling of a real RF mixer. With a WikiText-2 perplexity degradation of 4.0% on LLaMA-3.1-8B, AIR-LLM saves the energy by 157.7x/40.4x against the FP16 and weight-only quantization baselines; with 20 users, its airtime is 104.1x/26.0x shorter, respectively.

Zhihui Gao, Tingjun Chen, Dirk Englund

Massachusetts Institute of Technology Cambridge, MA, USA · Duke University Durham, NC, USA

On-Device Language Model InferenceLLM Inference AccelerationWireless Communications
AskPDF
Jul 19, 2026

OrderMoE: An expert similarity driven distributed edge MoE inference

Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it remains challenging to deploy MoE inference over resource-constrained and bandwidth-limited edge infrastructures. Existing distributed MoE serving methods mainly rely on exact expert placement, caching, replication, or communication scheduling, while overlooking the functional similarity among experts, which provides an opportunity to reduce cross-server token transmission. Therefore, this paper introduces a similarity-aware expert allocation and distributed deployment framework, dubbed OrderMoE, which aims to accelerate edge MoE inference while balancing inference latency, communication overhead, server workload, and inference quality. OrderMoE first constructs an expert similarity model based on router-induced logits representations and partitions experts in each MoE layer into multiple similarity groups. Then, it develops a similarity-aware expert grouping and deployment strategy to improve local similarity coverage across edge servers. Since reducing remote expert invocation and preserving exact inference quality are conflicting objectives, OrderMoE further designs a quality-aware and trajectory-aware runtime server-expert selection algorithm to decide whether a token should invoke its remote target expert or use a feasible local substitute expert. Experimental results on a real distributed edge testbed show that OrderMoE significantly reduces average latency, tail latency, cross-server traffic, and remote expert invocation ratio, while introducing only small and controllable inference quality degradation.

Xin Yuan, Ning Li, Quan Chen +2

School of Ocean Engineering, Harbin Institute of Technology, Heilongjiang, China · School of Computer Science and Technology, Harbin Institute of Technology, Heilongjiang, China · School of Computer Science and Technology, Guangdong University of Technology, Guangdong, China +2

Edge InferenceMixture-of-Experts Inference
AskPDF
Apr 25, 2026

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns

Most recent state-of-the-art (SOTA) large language models (LLMs) use Mixture-of-Experts (MoE) architectures to scale model capacity without proportional per-token compute, enabling higher-quality outputs at manageable serving costs. However, MoE inference at scale is fundamentally bottlenecked by expert load imbalance and inefficient token routing, especially in multi-node deployments where tokens are not guaranteed to be routed to local experts, resulting in significant inter-node all-to-all communication overhead. To systematically characterize these challenges, we profile SOTA open-source MoE models, including Llama 4 Maverick, DeepSeek V3-671B, and Qwen3-230B-A22B, on various datasets and collected over 100k real expert activation traces. Upon studying the expert activation patterns, we uncover various persistent properties across all the frontier MoE models: variable expert load imbalance, domain-specific expert activation where expert popularity shifts across task families (code, math, chat, general), and a strong correlation between prefill and decode expert activations. Motivated by these findings, we propose workload-aware micro-batch grouping and an expert placement strategy to maximize token locality to the destination expert, thereby reducing inter-node communication. Across models and datasets, these optimizations help reduce all2all communication data up to 20, resulting in lower MoE decode latency and better accelerator utilization.

Abhimanyu Bambhaniya, Geonhwa Jeong, Jason Park +6

1Georgia Institute of Technology, USA · 2Meta Platforms, Inc.

Expert Load BalancingMixture-of-Experts InferenceLLM RoutingMixture-of-Experts ModelsDistributed InferenceCommunication-Efficient Mixture-of-Experts
AskPDF