Joint Semantic-Channel Coding and Modulation for Generative Image Transmission

Hao Cao; Jingkai Ying; Yinhuan Huang; Zhijin Qin; Yuan Shen

cs.CVOct 21, 2026

Joint Semantic-Channel Coding and Modulation for Generative Image Transmission

Authors: Hao Cao, Jingkai Ying, Yinhuan Huang, Zhijin Qin, Yuan Shen

Organizations: Department of Electronic Engineering, Tsinghua University · State Key Laboratory of Space Network and Communications

Abstract

Digital image transmission typically optimizes source coding, channel coding, and modulation separately, limiting how source representations adapt to channel impairments. We propose SemQAM, a joint semantic-channel coding and modulation scheme for generative image transmission converting source to modulated symbols, such as standard QAM symbols. Channel impairments create two challenges: they can corrupt the latent representation at the receiver, and this corruption can compound through autoregressive coding, degrading reconstruction quality. First, SemQAM represents latent features with vector quantization (VQ), maps each VQ index to a QAM symbol, and updates the codeword corresponding to the detected symbol so that channel impairments directly affect codebook learning. Moreover, SemQAM uses the conditional expectation of received codewords to approximate the receiver reconstruction and reduce error propagation during autoregressive coding. SemQAM supports multiple SNRs and CBRs in a unified model, while also supporting conventional source coding through SemQAM-C. Experiments show that (i) under the same LDPC and QAM settings, SemQAM-C outperforms OneDC in LPIPS and DISTS with substantially lower complexity, showing that the unified model retains competitive capability for source coding. (ii) Compared with this separated counterpart, SemQAM reduces channel uses by 18.57% at matched LPIPS and outperforms the compared JSCC methods, demonstrating the benefit of joint coding.

Figures & tables

Panel of Figure 1 — Figure 1: Conventional separated transmission and SemQAM. (a) A conventional system separately optimizes source coding, channel coding, and modulation. (b) SemQAM is designed to jointly optimize end-to-end image transmission over a standard QAM constellation. SemQAM (i) trains VQ codewords using the detected indices at the receiver, and (ii) uses the expectation of received codewords to approximate the receiver reconstruction during autoregressive coding. $\mathbb{E}$ , VQ, and iVQ denote conditional expectation, vector quantization, and codeword lookup, respectively.

Figure 2: Rate and perceptual quality at 10 dB on four datasets. Lower values are better. FID vertical axes use base-2 logarithms.

	Computation				BD-rate $\downarrow$			Design
Method	Enc.	Dec.	FLOPs	Params	LPIPS	DISTS	FID	JSCC	Percep.	Mod.
MS-ILLM	165.38	147.79	599.52	181.40	252.63	286.55	198.99	—	$\checkmark$	—
OSCAR	53.04	167.56	6485.61	1009.30	91.81	96.05	60.47	—	$\checkmark$	—
OneDC	76.98	204.47	8557.18	1406.42	20.96	17.01	-12.00	—	$\checkmark$	—
DiT-IC	97.62	151.89	5743.15	1069.41	18.39	-11.98	-18.28	—	$\checkmark$	—
EF-LIC	17.62	13.72	279.61	35.74	12.38	14.06	16.51	—	$\checkmark$	—

Table 1: Results on Kodak at 10 dB, with latency measured on one NVIDIA A100. Latency is reported in ms, FLOPs in G, parameters in M, and BD-rate in percent. JSCC denotes joint semantic-channel coding, Percep. denotes generative optimization, and Mod. denotes joint optimization with modulation. SemQAM-C denotes the separated system combining the noiseless source codec with LDPC and QAM, while SemQAM denotes joint QAM transmission. SemQAM-C is the BD-rate anchor, and lower values are better. Bold and underline mark the best and second-best BD-rates.

Figure 3: QAM symbol usage across SNRs. Lower SNR concentrates usage on corner symbols, while higher SNR makes it more balanced.

Figure 4: LPIPS at CBR 0.005534 under different SNRs across four datasets. Lower is better.

Model	Ablation method					Computation			BD-rate (%)
Model	HP	AR	Deeper Backb.	E-RX	RX-CB	Enc.	Dec.	FLOPs	LPIPS
$\mathrm{M}_{0}$	—	—	—	$\checkmark$	$\checkmark$	2.31	3.77	310.29	0.00
$\mathrm{M}_{1}$	—	—	$\checkmark$	$\checkmark$	$\checkmark$	2.79	4.68	397.52	$-4.84$
$\mathrm{M}_{2}$	$\checkmark$	—	—	$\checkmark$	$\checkmark$	3.84	4.86	368.66	$-5.15$
$\mathrm{M}_{3}$	$\checkmark$	$\checkmark$	—	—	$\checkmark$	4.95	5.93	381.44	$-7.99$
$\mathrm{M}_{4}$	$\checkmark$	$\checkmark$	—	$\checkmark$	—	5.46	5.90	381.44	$>100$

Table 2: Ablation configurations on Kodak at 10 dB. HP, AR, Deeper Backb., E-RX, and RX-CB denote the hyperprior, quadtree autoregression, deeper backbone, expected received codewords, and receiver-side codebook loss, respectively.

\mathrm{M}_{0}

is the LPIPS BD-rate anchor, and lower is better.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Numerically optimized 1024-QAM symbol usage under the same average-power SNR normalization used by SemQAM. For each candidate distribution, the noise power is determined by its average transmitted symbol power, so concentrating probability on higher-energy symbols does not increase the operating SNR. Lower SNR produces stronger concentration on corner and boundary regions, while symbol usage becomes progressively broader as SNR increases.

Figure 6: Framework of SemQAM. The indices produced by RVQ are mapped to standard QAM constellation points and transmitted over the wireless channel.

SNR	MCS index	QAM order	Code rate	CBR from bpp $b$
10 dB	16	16	$658/1024$	$b/7.7109375$
16 dB	21	64	$616/1024$	$b/10.828125$
24 dB	28	64	$948/1024$	$b/16.6640625$

Appendix

Table 3: Modulation and coding settings used for the separated systems. The 16 and 24 dB settings satisfy a 1% BLER threshold in the reference curves.

Metric	Family	Method	Kodak	Tecnick	DIV2K	CLIC
LPIPS	JSCC	NTSCC	473.89	484.29	453.53	—
		NTSCC+	438.79	472.61	413.23	914.55
		MDJCM-A	443.14	446.27	396.04	845.35
		Quad-JSCC	67.60	35.57	45.29	53.43
	GIC	MS-ILLM	252.63	143.97	195.83	227.06
		OneDC	20.96	-1.14	15.00	15.99

Appendix

Table 4: 10 dB BD-rate (%) relative to SemQAM-C. Lower is better. Bold and underline mark the two best results. A dash denotes an unavailable comparison.

Figure 7: Additional comparison at 16 dB. Rows are LPIPS, DISTS, and FID, and columns are Kodak, Tecnick, DIV2K, and CLIC 2020 Professional. Separated codecs use MCS 21 and ideal successful decoding. FID is displayed on a base-2 logarithmic axis.

Figure 8: Additional comparison at 24 dB with the same metric and dataset arrangement as Fig. 7 . Separated codecs use MCS 28 and ideal successful decoding. SemQAM uses the codebooks for the corresponding 24 dB operating points.

	1080 $\times$ 1920			1440 $\times$ 2560			2160 $\times$ 3840
Method	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.
NTSCC	130.97	138.18	7.24	245.81	371.34	12.95	1034.13	1368.10	28.55
NTSCC+	222.02	181.14	3.96	395.41	320.84	6.85	849.50	676.75	14.28
MDJCM-A	141.46	279.99	7.69	250.12	499.22	13.57	753.93	1103.39	28.69
Quad-JSCC	158.77	267.91	4.98	273.26	586.64	13.18	597.85	1076.13	13.58
MS-ILLM	350.85	379.01	2.99	516.47	601.18	4.87	1305.93	1613.82	9.94

Appendix

Table 5: Higher resolution latency and peak GPU memory on A100. Enc. and Dec. are in ms and peak memory is in GiB. SemQAM averages all sixteen CUDA operating points. OSCAR exceeds 80 GiB at 4K.

Figure 9: Visual comparison with previous methods. LPIPS and CBR are reported for each reconstruction, and the marked regions are enlarged for comparison.

Figure 10: Additional visual comparison with previous methods. LPIPS and CBR are reported for each reconstruction, and the marked regions are enlarged for comparison.

Joint Source-Channel Coding

See all 17 papers

JSCGC: Joint Source-Channel-Generation Coding for Wireless Generative Communications
Jun 11, 2026Tong Wu, Zhiyong Chen, Guo Lu +4Joint Source-Channel Coding Wireless Communications
Cooperative Medianet Innovation Center, the School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission
Jun 4, 2026Tianqi Ren, Rongpeng Li, Xianfu Chen +2Learned Image Compression Diffusion Language Models
College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China · Shenzhen CyberAray Network Technology Co., Ltd, Shenzhen 518000, China · School of Mechanical Engineering and Electronic Information, China University of Geosciences, Wuhan 430074, China +1
When Semantic Communication Meets Queueing: Cross-Layer Latency and Task Fidelity Optimization
May 6, 2026Yalin E. Sagduyu, Tugba ErpekSemantic Communication Age of Information
Nexcepta, Gaithersburg, MD, USA

Jun 11, 2026J/K move · Enter open · S save

JSCGC: Joint Source-Channel-Generation Coding for Wireless Generative Communications

Tong Wu, Zhiyong Chen, Guo Lu, Li Song+3

Cooperative Medianet Innovation Center, the School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Joint Source-Channel Coding Wireless Communications Conditional Generative Modeling

Open paper AskPDF arXiv

Conventional communication systems, including both separation-based coding and learning-based joint source-channel coding (JSCC), are typically designed under Shannon's rate-distortion theory. However, relying on generic distortion metrics fails to capture complex human visual perception, often resulting in blurred or unrealistic reconstructions. In this paper, we propose Joint Source-Channel-Generation Coding (JSCGC), a generative communication paradigm that replaces the conventional decoder with a generative model at the receiver. The received signal is treated as a condition that controls the sampling process into the learned conditional distribution, reformulating communication from deterministic reconstruction for distortion minimization to controlled generation for mutual information maximization under perceptual constraints. Based on this formulation, we develop a unified joint training and efficient stochastic sampling framework, and provide theoretical analysis of its effectiveness in both learning and inference stages. Extensive experiments on latent-space image transmission demonstrate that the JSCGC consistently improves feature-based, semantic-level, and distributional quality across diverse channel conditions, while exhibiting a distinct error behavior characterized by semantic inconsistency rather than distortion.

Abstract

Joint Source-Channel Coding

Explore similar work

JSCGC: Joint Source-Channel-Generation Coding for Wireless Generative Communications

Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission

When Semantic Communication Meets Queueing: Cross-Layer Latency and Task Fidelity Optimization