ZeroCode: On-demand Error-Correcting Code Construction from the Zero Matrix via Reinforcement Learning

Ju-Hyeong Lee; Yongjune Kim; Sang-Hyo Kim; Dae-Young Yun; Hee-Youl Kwak

cs.LGSep 28, 2026

ZeroCode: On-demand Error-Correcting Code Construction from the Zero Matrix via Reinforcement Learning

Authors: Ju-Hyeong Lee, Yongjune Kim, Sang-Hyo Kim, Dae-Young Yun, Hee-Youl Kwak

Organizations: University of Ulsan · Pohang University of Science and Technology (POSTECH) · Sungkyunkwan University

Abstract

Error-correcting codes (ECCs) are essential across diverse applications, from wireless communications and storage to quantum computing, yet each application imposes distinct design requirements on the parity-check matrix (PCM). To address these on-demand requirements in a unified framework, we propose ZeroCode, a reinforcement learning (RL)-based approach that constructs PCMs sequentially from the all-zero matrix. ZeroCode formulates construction as a discrete sequential decision-making problem and uses proximal policy optimization with action masking to select valid edges. ZeroCode achieves a gain of approximately 1 dB over the prior RL-based construction method at a bit error rate (BER) of $10^{-4}$ for the (32,16) code and outperforms existing genetic, differentiable, and classical code-design methods in our experiments. Beyond optimizing decoding performance, the masking mechanism allows on-demand structural constraints, such as a maximum degree, 4-cycle-free structure, and quasi-cyclic structure, to be flexibly incorporated. Moreover, a single policy rollout yields a library of PCMs with varying edge counts, offering trade-offs between decoding performance and complexity without retraining. Overall, ZeroCode addresses diverse code-design requirements within a unified framework, providing solutions with optimized decoding performance under given constraints.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of (a) ZeroCode compared with (b) the prior RL-based approach, DDPG ( Tian et al., 2025 ) . DDPG starts from a pre-constructed PCM and updates multiple edges at each step, whereas ZeroCode begins with an all-zero matrix and the agent incrementally adds a single edge per step. In both methods, the PCM at each step serves as the state, and the BER obtained from the decoder is used as the reward signal.

Figure 2: Overview of the PCM construction process in ZeroCode. The agent receives the current PCM as the state, applies an action mask to retain only edge candidates that satisfy the structural constraints, and samples a single edge from the masked policy. The updated matrix is evaluated by the BP decoder to produce the reward signal. Starting from the all-zero matrix, ZeroCode repeats this process to optimize PCMs.

Panel of Figure 3 — Figure 3: Comparison of the codes constructed by ZeroCode with DDPG ( Tian et al., 2025 ) , GA ( Elkelesh et al., 2019 ) , DL ( Choukroun and Wolf, 2024b ) , and the classical construction ( Richardson et al., 2001 ) for various code dimensions over the AWGN channel: (a) $(32,16)$ , (b) $(64,32)$ , (c) $(128,64)$ , and (d) $(256,128)$ .

Method	RL Algorithm	Initial PCM	Edge Update	BER ( ${\rm E}_{\rm b}/{\rm N}_{0}=5.0$ dB)
(A) ( Tian et al., 2025 )	DDPG	CCSDS	Multi-Flip	$1.0\times 10^{-3}$
(B)	PPO	CCSDS	Multi-Flip	$2.1\times 10^{-3}$
(C)	PPO	CCSDS	Single-Flip	$1.0\times 10^{-3}$
ZeroCode	PPO	All-zero	Single-Add	$\mathbf{2.2\times 10^{-4}}$

Table 1: Ablation study of ZeroCode on the

(32,16)

code.

Figure 4: Sequential construction of a code library using a single trained ZeroCode policy for the

(32,16)

code. The first matrix shows the all-zero initialization, while the remaining matrices show valid PCMs obtained along the construction trajectory, together with their step indices and the required

({\rm E}_{\rm b}/{\rm N}_{0})_{\rm req}

for achieving

\mathrm{BER}=10^{-3}

Panel of Figure 5 — Figure 5: (a) Performance–complexity trade-off achieved by a single ZeroCode policy on the $(32,16)$ code. (b) The same trade-off on the $(64,32)$ code, obtained by applying degree masks $d_{\max}\in\{4,6\}$ at inference time to the trained policy. (c) BER comparison between the 5G LDPC code and ZeroCode designed under the raptor-like rate-compatible structural constraint. In (a)–(b), the y-axis represents the required ${\rm E}_{\rm b}/{\rm N}_{0}$ to achieve $\mathrm{BER}=10^{-3}$ .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Learning rate	$10^{-5}$
Total timesteps $T$	$3\times 10^{6}$
Rollout size $N$	$2{,}048$
Number of epochs $K$	$10$
Mini-batch size $b$	$64$
Discount factor $\gamma$	$0.99$

Appendix

Table 2: Hyperparameters used to train ZeroCode.

Method	Cost for a single PCM	Cost for a library ( $L=41$ )
ZeroCode	4 hours ( $t_{a}$ )	4 hours ( $t_{a}$ )
GA ( Elkelesh et al., 2019 )	6 hours ( $t_{b}$ )	246 hours ( $Lt_{b}$ )
DL ( Choukroun and Wolf, 2024b )	2 hours ( $t_{c}$ )	82 hours ( $Lt_{c}$ )

Appendix

Table 3: Approximate wall-clock training cost for the

(32,16)

code. The code library contains the

L=41

valid PCMs from the first valid design at 36 edges to the best design at 76 edges. For GA and DL, library costs are estimated as

L

times the single-PCM cost, assuming independent optimization for each target edge count.

Figure 6: Training robustness of ZeroCode across eight random seeds for the

(32,16)

code. The solid red curve shows the mean best-PCM reward

R_{\mathrm{best}}=-\ln(\mathrm{BER}_{\mathrm{best}})

across seeds. The shaded region denotes the 95% confidence interval across eight random seeds. Rewards are evaluated at

{\rm E}_{\rm b}/{\rm N}_{0}=5.0

dB and averaged over non-overlapping bins of 200 episodes. The horizontal dashed line denotes the reward of the final PCM produced by DDPG ( Tian et al., 2025 ) under the same evaluation setting.

Panel of Figure 7 — Figure 7: Comparison of three actor–critic architectures (GNN, GNN+MLP, and MLP) on the $(32,16)$ code. (a) Training curves show the 100-episode moving average of the reward $R_{\mathrm{best}}=-\ln(\mathrm{BER}_{\mathrm{best}})$ of the best PCM encountered in each episode. (b) Performance of the resulting codes constructed by each architecture.

$(n,k)$	Action mask	Edges	$d_{\max}$	4-cycle	6-cycle	BER
$(32,16)$	4-cycle	76	5	0	88	$\mathbf{1.81\times 10^{-4}}$
$(32,16)$	Max-degree ( $d_{\max}=8$ )	76	5	3	99	$2.24\times 10^{-4}$
$(64,32)$	4-cycle	178	6	0	311	$6.76\times 10^{-5}$
	Max-degree ( $d_{\max}=8$ )	176	7	5	296	$7.25\times 10^{-5}$
	QC ( $Z=8$ )	168	4	0	144	$\mathbf{4.23\times 10^{-5}}$
$(128,64)$	4-cycle	400	8	0	818	$1.15\times 10^{-5}$

Appendix

Table 4: Comparison of action masking strategies. BER is evaluated at

{\rm E}_{\rm b}/{\rm N}_{0}=5.0

dB using a BP decoder; the lowest BER for each block length is shown in bold.

Panel of Figure 8 — Figure 8: Generality of ZeroCode: (a) BER comparison between ZeroCode and the classical LDPC construction over a Rayleigh fading channel; (b) FER comparison of the ZeroCode polar-code construction with AI Coding, PW, and DE/GA over the AWGN channel.

Explore similar work

MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes
Sep 1, 2026Jonggyu Jang, Hongjae Nam, Vishrant Tripathi +2Source-Channel Coding Error Correction
Department of Electronics Engineering, Chung-nam National University, Daejeon 34134, Republic of Korea · Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907 USA · Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea
Learning to Concatenate Quantum Codes
Apr 16, 2026Nico Meyer, Christopher Mutschler, Dominik Seuß +2Quantum Error Correction Post-Quantum Cryptography
Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, Nuremberg, Germany · Machine Learning and Positioning Systems Lab, University of Technology Nuremberg (UTN), Nuremberg, Germany · Center for Artificial Intelligence (CAIRO), Technical University of Applied Sciences Würzburg-Schweinfurt, Germany +1
Score Based Error Correcting Code Decoder
May 27, 2026Alon Helvits, Eliya NachmaniQuantum Error Correction Signal-To-Noise Ratio
School of Electrical and Computer Engineering (ECE), Ben Gurion University, Beer Sheva, Israel.

Sep 1, 2026, cs.ITJ/K move · Enter open · S save

MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes

Jonggyu Jang, Hongjae Nam, Vishrant Tripathi, David J. Love+1

Department of Electronics Engineering, Chung-nam National University, Daejeon 34134, Republic of Korea · Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907 USA · Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea

Source-Channel Coding Error Correction Intrinsic Encoders Transformer Architectures Ai-Assisted Programming Tasks

Open paper AskPDF arXiv

Feedback-based coding schemes have demonstrated substantial performance gains over today's open-loop coding schemes. Unfortunately, these gains are usually achieved in idealized settings with perfect feedback. Over the last few years, machine learning-based schemes have been shown to be promising solutions for implementing feedback-based codes, particularly when combined with short-block-length open-loop error correcting codes (ECCs) in a concatenated coding structure. However, existing ML-based feedback schemes remain agnostic to the outer code's structure, potentially misallocating feedback resources on error patterns already correctable by the outer ECC. To address this, we propose MaskCode, a Transformer-based inner feedback code for concatenated coding systems, which explicitly incorporates structural knowledge of the outer linear block code into the inner feedback encoder design via two synergistic mechanisms: 1) a soft syndrome-based input that informs the encoder about potential parity constraint violations, and 2) a code-aware attention mask derived from the Tanner graph. We further show that end-to-end training with a differentiable belief propagation (BP) decoder offers no additional gain, as MaskCode's structure-aware design already internalizes the structural knowledge of the outer code; in fact, backpropagation through the iterative BP decoder introduces gradient explosion, which degrades rather than improves performance. Extensive evaluations on BCH and LDPC outer codes demonstrate that MaskCode consistently outperforms all baselines, achieving up to 1.5 dB SNR gain.

Abstract

Explore similar work

MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes

Learning to Concatenate Quantum Codes

Score Based Error Correcting Code Decoder