CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference

Hai Chi; Junrui Zhang; Rui Ning; Chonggang Wang; Robert Gazda; Huanrui Yang; Hongyi Wu

cs.LGAug 22, 2026

CST: Collaborative Selective Transmission for Communication-Efficient Multimodal Edge Inference

Authors: Hai Chi, Junrui Zhang, Rui Ning, Chonggang Wang, Robert Gazda, Huanrui Yang, Hongyi Wu

Organizations: Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, USA · Department of Computer Science, Old Dominion University, Norfolk, VA, USA · InterDigital, Conshohocken, PA, USA

Abstract

Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensional helper representations incurs substantial communication overhead and can lead to high end-to-end latency. Existing communication-efficient methods reduce payloads through compression, semantic coding, or feature selection, yet typically optimize compactness or task relevance without explicitly accounting for information already represented at the main device. Consequently, task-relevant but redundant helper features may still consume bandwidth. We present Collaborative Selective Transmission (CST), a main-directed query--response framework that retrieves only helper information complementary to the current main representation. Inspired by Partial Information Decomposition and the Multiview Redundancy Assumption, CST learns sample-adaptive, helper-specific sparse retrieval supports while discouraging retrieval of semantics already covered by the main device or duplicated across helpers. During inference, the main device transmits only support indices, and each helper returns the corresponding latent values, avoiding dense helper-feature exchange. Across three real-world multimodal sensing benchmarks, CST transmits no more than 14.18% of helper feature values while achieving best or near-best task performance among the evaluated methods. Experiments on a five-node NVIDIA Jetson Orin Nano testbed across 5--100 Mbps demonstrate up to a $4.27\times$ speedup over Transmit-All in end-to-end inference, confirming practical end-to-end latency reductions.

Figures & tables

Fig. 1: Collaborative multimodal transmission with three representative MM-Fi [ 36 ] helpers in a 3D pose estimation example. (a) Transmit-All sends complete helper representations. (b) Representative methods use local selection, learned compression, or request-guided helper-side selection; the paths denote distinct method families. (c) CST generates helper-specific sparse supports from the current main representation and retrieves the corresponding values.

Fig. 2: Latency profiling on MM-Fi [ 36 ] in a network under a shared aggregate bandwidth of 5 Mbps, with RTT and jitter configured to 30 ms and 5 ms, respectively, and a batch size of 32. Communication dominates the end-to-end latency in all shown baseline methods, including Transmit-All, VDDIB-SR [ 26 ] , SINFONY [ 1 ] , and How2Comm [ 35 ] , whereas CST achieves a total latency of 7.71 s.

Fig. 3: Overview of CST, illustrated with an MM-Fi 3D pose estimation example using RGB as the main modality and LiDAR and WiFi-CSI as two representative helpers. The main representation generates a shared training-time redundancy anchor and helper-specific complementary anchors. During inference, the main device sends only the support indices of each complementary anchor, and the helpers return the requested latent values for residual fusion and task prediction. Additional helpers follow the same query–response path.

Fig. 4: Qualitative RGB-helper selection on DarkAct with Thermal as the main modality. Highlighted locations denote retained features. CST uses a smaller support concentrated on informative human and scene regions.

Dataset	Task	Modalities	Seq. Length ( $G_{i}$ )	Metrics
MM-Fi [ 36 ]	3D Pose Estimation	RGB , Depth, LiDAR, mmWave, WiFi-CSI	32, 180, 32, 32, 32	MPJPE, PA-MPJPE
CUHK-S [ 12 ]	Action Recognition	Skeleton , Depth/Color, IR, IMU, Radar	32, 196, 196, 128, 20	Acc., F1, Prec., Rec.
DarkAct [ 29 ]	Action Recognition	Thermal , RGB	32, 32	Accuracy

TABLE I: Datasets, modality assignments, sequence lengths, and task-specific performance metrics. Main modalities are bold. Metric definitions follow the benchmark papers.

Method	MM-Fi			CUHK-S					DarkAct
Method	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	Tx (%) $\downarrow$	Acc. (%) $\uparrow$	F1 (%) $\uparrow$	Prec. (%) $\uparrow$	Rec. (%) $\uparrow$	Tx (%) $\downarrow$	Acc. (%) $\uparrow$	Tx (%) $\downarrow$
Main-Only	119.20	68.94	–	66.28	63.43	65.50	63.90	–	64.55	–
Transmit-All	87.35	37.06	100.00	67.11	63.99	65.18	64.83	100.00	67.84	100.00
BottleNet++ [ 27 ]	86.13	38.71	50.00	68.29	66.78	69.96	67.74	50.00	66.45	50.00
VIB-Dyn [ 25 ]	87.72	44.28	30.66	72.82	68.52	69.69	69.90	30.86	66.35	30.66
VDDIB-SR [ 26 ]	86.39	38.76	31.21	70.81	67.29	68.64	68.89	30.82	66.45	45.34

TABLE II: Task performance and transmission rate on three benchmarks. Tx reports the transmitted representation size as a percentage of the complete helper representations. MM-Fi errors are in millimeters; CUHK-S F1, precision, and recall are macro-averaged. Actual bidirectional traffic for representative methods measured on the testbed is reported separately in Table III .

\uparrow

and

\downarrow

indicate whether higher or lower values are preferred, respectively. Best observed values are bold.

Method	MM-Fi	CUHK-S	DarkAct
Transmit-All	565.59	1106.30	65.65
BottleNet++	282.97	553.33	32.84
VIB-Dyn	173.67	341.65	20.17
VDDIB-SR	172.68	341.49	29.97
SINFONY	282.97	553.33	32.84
DynamicViT	189.91	482.26	36.05

TABLE III: Mean bidirectional traffic (kilobytes per sample) on the testbed, including requests, responses, framing, and metadata.

Fig. 5: Mean per-sample end-to-end latency, computed as the mean complete-batch latency divided by the batch size of 32, versus shared aggregate bandwidth on MM-Fi and CUHK-S. RTT and jitter are configured to 30 ms and 5 ms, respectively.

Method	10 ms		30 ms		50 ms		100 ms
	Mean	p95	Mean	p95	Mean	p95	Mean	p95
Transmit-All	8.95	9.31	8.76	8.78	8.80	8.83	8.88	8.92
SINFONY	5.03	5.37	4.99	4.99	4.99	5.01	5.12	5.20
DynamicViT	3.78	4.13	3.89	4.27	3.87	4.31	3.99	4.39
How2Comm	5.31	5.71	5.36	5.75	5.41	5.78	5.62	6.01
CST	2.86	2.89	3.05	3.10	2.97	3.00	3.26	3.29

TABLE IV: Mean and p95 complete-batch end-to-end latency on MM-Fi under varying configured RTTs. The bandwidth is 10 Mbps, the jitter target is 5 ms, and the batch size is 16. Values are in seconds and computed over 20 measured batches.

Fig. 6: Mean complete-batch end-to-end latency versus batch size on MM-Fi and CUHK-S. The shared aggregate bandwidth is 10 Mbps, with configured RTT and jitter targets of 30 ms and 5 ms, respectively.

Variant	Acc. $\uparrow$	F1 $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$	Tx (%) $\downarrow$
Full CST	72.99	68.59	70.81	69.10	8.66
Direct- $A_{c}$	72.32	67.90	70.79	68.59	51.82
Direct- $A_{c}$ + $\mathcal{L}_{\mathrm{aux}}$	72.48	68.27	70.72	68.90	50.79
w/o $\mathcal{L}_{\mathrm{sim}}$	72.31	67.57	70.42	67.85	42.34
w/o $\mathcal{L}_{\mathrm{ortho}}$	72.32	68.03	70.72	69.04	36.62
w/o $\mathcal{L}_{\mathrm{aux}}$	72.82	68.30	70.17	68.91	37.26

TABLE V: CUHK-S ablation results. Direct-

A_{c}

uses only

\mathcal{L}_{\mathrm{task}}

and

\mathcal{L}_{\mathrm{sparsity}}

; the second variant retains

\mathcal{L}_{\mathrm{aux}}

while removing the redundancy branch. All values are percentages, and best values are bold.

Method	$U_{\mathrm{H}}$	$S$	$C=U_{\mathrm{H}}+S\uparrow$	Tx (%) $\downarrow$
Transmit-All	$1.783\pm 0.078$	$1.464\pm 0.057$	$3.247\pm 0.125$	$100.00$
BottleNet++	$1.602\pm 0.044$	$1.406\pm 0.046$	$3.008\pm 0.078$	$50.00$
VIB-Dyn	$1.576\pm 0.047$	$1.403\pm 0.047$	$2.979\pm 0.078$	$30.66$
SINFONY	$1.651\pm 0.111$	$1.375\pm 0.078$	$3.026\pm 0.166$	$50.00$
DT-JSCC	$1.659\pm 0.133$	$1.363\pm 0.077$	$3.023\pm 0.197$	$50.00$
DynamicViT	$1.712\pm 0.154$	$1.290\pm 0.088$	$3.002\pm 0.216$	$56.03$

TABLE VI: Post-hoc Flow-PID results on DarkAct, reported in bits as mean

\pm

standard deviation over three fixed Flow initializations. Upward and downward arrows indicate that higher and lower values are preferred, respectively. The largest mean

C

and lowest Tx among communication-efficient methods are bold.

Communication-Efficient Collaborative Perception

Communication-efficient collaborative perception studies how networked agents share useful sensor information with minimal bandwidth while improving their joint view of the scene.

See all 26 papers

EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression
Sep 10, 2026Motahare Mounesan, Irfan KhanEfficient Multimodal Inference Edge-Cloud Collaborative Inference
Texas A&M University
Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference
Sep 29, 2026Luning Pang, Cheng Yuan, Jiawei Shao +2Efficient Multimodal Inference Visual Token Compression
Department of Electronic Engineering, and Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China · Institute of Artificial Intelligence (TeleAI), China Telecom, Beijing 100033, China
Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference
Apr 10, 2026Yueyuan Sui, Payal Mohapatra, Doğaç Eldenk +5Efficient Multimodal Inference Neural Network Pruning
Northwestern University, Evanston, Illinois, USA

Sep 10, 2026J/K move · Enter open · S save

EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression

Motahare Mounesan, Irfan Khan

Texas A&M University

Efficient Multimodal Inference Edge-Cloud Collaborative Inference Edge Computing Multimodal Large Language Models Multimodal Token Compression Edge-Cloud Computing

Open paper AskPDF arXiv

Recent advances in multimodal large language mod- els (MLLMs) have opened new opportunities for edge intelligence by enabling reasoning across heterogeneous sensor modalities, such as vision, text, and telemetry data. However, deploying these capabilities on resource-constrained edge platforms remains challenging due to the substantial computational, memory, and communication demands of modern MLLMs. Rather than transmitting raw sensor observations or partitioning neural networks at intermediate layers, Edge Multi-Modal Intelligence (EMMI) communicates a compact representation between edge devices and server resources, enabling communication-efficient edge MLLM inference. To achieve this, EMMI performs modality-specific encoding, cross-modal representation fusion, and learned compression at the edge, transmitting only a compact latent representation to server-side resources for high-capacity MLLM reasoning. This representation-centric design reduces communication overhead, preserves local data privacy, and provides a fixed-size interface between heterogeneous edge devices and server-side MLLMs. Evaluation on a representative multimodal benchmark demonstrates that EMMI can reduce the communication payload by 32x while maintaining comparable downstream accuracy, resulting in up to a 3.4x reduction in estimated end-to-end inference latency under bandwidth-constrained edge conditions.

Abstract

Communication-Efficient Collaborative Perception

Explore similar work

EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression

Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference

Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference