Masked diffusion language models generate tokens in parallel, and the order in which tokens are unmasked affects final-answer accuracy. Several methods learn unmasking policies from task rewards, but these policies cannot remask earlier predictions. Remasking lets the model correct those predictions using newly generated context. Heuristic rules guide both unmasking and remasking, while supervised remaskers learn to detect token errors. Neither approach directly optimizes these decisions for final-answer correctness. To address this gap, we introduce ReMaRL (Remasking with Reinforcement Learning), a lightweight policy that learns which tokens to unmask, retain, or remask. We train this roughly 0.3M-parameter policy with Group Relative Policy Optimization (GRPO) from final-answer rewards while keeping the diffusion model frozen. With LLaDA-8B-Instruct, ReMaRL achieves the highest scores among the compared unmasking and remasking policies within the tested inference budgets of 4-32 model calls on math and 8-16 calls on code.
Figures & tables
Figure 1
Figure 1: Math accuracy versus inference budget. The solid blue line connects ReMaRL policies trained for 32k groups and evaluated at their respective training budgets. The dash-dot curve reuses the NFE64 checkpoint at larger budgets. Jazbec et al. A and UPO U results are taken from their papers; UPO uses 128-token answers.
Figure 2: Code pass@1 versus inference budget for selected methods. The solid blue line connects three ReMaRL policies trained for 32k groups, one at each budget of 8, 16, and 32 calls. Each is evaluated at its training budget. The dash-dot ReMaRL curve reuses the NFE32 checkpoint at larger budgets. Full-diffusion and semi-autoregressive results are grouped in separate legend columns. Results marked A are taken from Jazbec et al. (2025) .
Figure 3: Math checkpoints trained for 32k groups, reevaluated at different inference budgets. Dot-centered diamonds mark matching training and inference budgets.
Variant
GSM8K
MATH500
ReMaRL
50.04
20.00
No position embedding
48.98
19.73
No hidden states
45.08
18.20
No remasking
38.26
15.00
Position and mask biases only
29.16
14.07
Random init., no added confidence
47.54
18.20
Table 1: Math accuracy (%) at eight calls after 16k accepted training groups.
Figure 4: Training-data size at NFE8 and 16k training groups. Dashed blue curves track the three seeds across sizes; each seed controls both subset sampling and training. The darker curve is their mean.
Math
Code
8 calls
16 calls
8 calls
16 calls
P2 ′ temperature
GSM8K
MATH500
GSM8K
MATH500
HumanEval
MBPP
HumanEval
MBPP
0
23.50
13.60
42.68
19.00
14.02
3.00
17.68
11.40
0.05
29.19
14.80
41.57
16.87
13.01
4.60
18.70
9.33
0.1
34.04
16.00
46.25
18.00
13.41
9.27
18.90
17.73
0.25
42.30
17.87
56.25
20.93
14.23
21.47
22.56
29.33
Table 2: P2 ′ position-temperature sweeps and ReMaRL on the full benchmarks (%). Here τ=0 denotes greedy position selection. Gray shading marks the temperature selected separately for each domain on training questions at eight calls and used unchanged at both budgets. Bold marks the highest score in each column.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Mean-NFE budget
Method
Blocks
4
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
36.14
53.60
65.13
72.81
79.71
—
—
ReMaRL (ours), trained at NFE64
1×256
—
—
—
—
79.71
80.39
82.49
P2 ′ (ours, τ=0.25 )
1×256
27.29
42.30
56.25
63.79
66.87
69.12
69.42
P2 ′ (ours, τ=1 )
1×256
16.17
28.76
43.59
59.19
68.11
70.58
73.16
Appendix
Table 3: GSM8K accuracy (%). Adaptive cells show the best observed test score within each mean-NFE budget, with mean NFE below. Bars scale from zero to the maximum score in each column; bold marks that score. Unmarked: computed here; A : Jazbec et al.; U : UPO; S : SAS; D : DCoLT ( Huang et al., 2025a , Table 4; published prompt/scorer). NR: block size unspecified. Dashes indicate unavailable results.
Mean-NFE budget
Method
Blocks
4
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
17.07
19.00
24.87
27.27
31.07
—
—
ReMaRL (ours), trained at NFE64
1×256
—
—
—
—
31.07
32.93
33.27
P2 ′ (ours, τ=0.25 )
1×256
13.73
17.87
20.93
22.60
24.67
26.27
26.07
P2 ′ (ours, τ=1 )
1×256
11.07
13.73
17.73
21.27
24.40
27.07
27.20
Appendix
Table 4: MATH500 comparison. Accuracy (%) under mean-NFE budgets. Adaptive rows report the best observed test score; actual mean NFE is in parentheses. Bars scale from zero to the column maximum; bold marks the highest score. Superscripts identify results from Jazbec et al. A , UPO U , and SAS S . Dashes denote unmeasured budgets or no eligible operating point.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
20.73
29.47
31.71
—
—
—
ReMaRL (ours), trained at NFE32
1×256
—
—
31.71
38.41
38.62
39.23
P2 ′ (ours, τ=0.25 )
1×256
14.23
22.56
24.19
23.17
25.20
22.97
P2 ′ (ours, τ=1 )
1×256
11.59
14.84
19.92
26.22
22.36
24.80
Appendix
Table 5: HumanEval comparison. Pass@1 (%) under mean-NFE budgets. Adaptive rows report the best observed test score; actual mean NFE is in parentheses. Bars scale from zero to the column maximum; bold marks the highest score. Scores marked A are from Jazbec et al. Dashes denote unmeasured budgets or no eligible operating point.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
25.40
32.40
36.47
—
—
—
ReMaRL (ours), trained at NFE32
1×256
—
—
36.47
39.53
41.33
43.53
P2 ′ (ours, τ=0.25 )
1×256
21.47
29.33
33.67
33.73
37.00
35.73
P2 ′ (ours, τ=1 )
1×256
15.73
23.33
28.07
32.27
32.47
35.93
Appendix
Table 6: MBPP comparison. Pass@1 (%) under mean-NFE budgets. Adaptive rows report the best observed test score; actual mean NFE is in parentheses. Bars scale from zero to the column maximum; bold marks the highest score. Superscripts identify results from Jazbec et al. A and SAS S . Dashes denote unmeasured budgets or no eligible operating point.
Figure 5: Original P2 and P2 ′ with greedy selection on the full test sets. P2 scores each fresh proposal; P2 ′ scores the current token at unmasked positions.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
47.86
58.15
65.48
—
—
—
P2 ′ (ours, τ=1 )
1×256
20.29
38.51
50.72
57.42
60.73
60.58
P2 ′ (ours, greedy)
1×256
2.20
9.63
27.29
38.82
39.58
35.71
Confidence
1×256
1.59
8.57
32.30
42.99
33.59
35.78
Appendix
Table 7: Dream GSM8K accuracy (%). Adaptive cells show the best observed test score within each mean-NFE budget, with actual mean NFE below. All scores are computed here. Bars scale from zero to the column maximum; bold marks the highest score. Dashes indicate an unmeasured budget or no eligible operating point.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
18.60
24.33
27.40
—
—
—
P2 ′ (ours, τ=1 )
1×256
8.73
16.00
20.67
22.80
22.27
21.53
P2 ′ (ours, greedy)
1×256
0.60
3.80
12.80
19.80
19.20
21.60
Confidence
1×256
1.00
3.80
15.00
21.00
17.20
21.20
Appendix
Table 8: Dream MATH500 accuracy (%). Adaptive cells show the best observed test score within each mean-NFE budget, with actual mean NFE below. All scores are computed here. Bars scale from zero to the column maximum; bold marks the highest score. Dashes indicate an unmeasured budget or no eligible operating point.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
24.39
38.01
46.95
—
—
—
P2 ′ (ours, τ=1 )
1×256
10.16
19.31
26.22
30.69
31.71
33.94
P2 ′ (ours, greedy)
1×256
7.32
14.02
18.90
28.05
29.27
33.54
Confidence
1×256
9.15
11.59
21.34
28.66
28.66
26.22
Appendix
Table 9: Dream HumanEval pass@1 (%). Adaptive cells show the best observed test score within each mean-NFE budget, with actual mean NFE below. All scores are computed here. Bars scale from zero to the column maximum; bold marks the highest score. Dashes indicate an unmeasured budget or no eligible operating point.
Mean-NFE budget
Method
Blocks
8
16
32
64
128
256
Fixed-budget decoding
ReMaRL (ours)
1×256
29.27
40.13
51.87
—
—
—
P2 ′ (ours, τ=1 )
1×256
23.60
34.07
41.87
45.93
47.33
49.53
P2 ′ (ours, greedy)
1×256
0.60
9.20
23.40
39.80
45.40
45.60
Confidence
1×256
1.40
6.60
22.80
44.40
45.80
47.60
Appendix
Table 10: Dream MBPP pass@1 (%). Adaptive cells show the best observed test score within each mean-NFE budget, with actual mean NFE below. All scores are computed here. Bars scale from zero to the column maximum; bold marks the highest score. Dashes indicate an unmeasured budget or no eligible operating point.
Figure 6: Generation trajectories on GSM8K at NFE32. (a) Visible non-special tokens in the first 64 positions; (b) mean position of all currently unmasked tokens, excluding special tokens; (c) remasking events per call. Curves show averages over questions and inference seeds.
Method
Remasking events
Replacements
Replaced (%)
ReMaRL
673
33
4.9
P2 ′ , τ=0.25
1,158
27
2.3
P2 ′ , τ=1
1,227
84
6.8
Appendix
Table 11: Mean remasking events and token replacements per answer, rounded to whole counts. The last column gives the percentage of remasking events that lead to a token replacement.
Figure 7: Math training and full-test histories. Each color denotes a policy trained at a different inference budget. The top panels show smoothed training accuracy; the bottom panels show test evaluations.
Figure 8: Code training and full-test histories for NFE8, NFE16, and NFE32. Main comparisons use each run’s 32k-group checkpoint.
Figure 9: Full-test learning curves for the NFE8 ablation variants.
Figure 10: Full-test accuracy at NFE8 for ReMaRL and three P2 ′ settings: the default position temperature, a tuned position temperature, and a retention bonus tuned to match the remasking frequency of the 32k-group ReMaRL policy.
Figure 11: Remasking frequency through 32k training groups. Curves are trailing means over up to 128 collections, weighted by the number of attempted trajectories and including trajectories from rejected groups.
Position temperature τ
GSM8K accuracy (%)
0
33.36
0.10
34.47
0.25
39.20
0.50
48.19
0.75
51.98
0.85
53.53
Appendix
Table 12: Independent position-temperature reevaluation of the fixed NFE8 ReMaRL policy trained for 32k groups. At τ=0 , position selection is greedy. Token predictions are greedy throughout.
Figure 12: Training accuracy for the NFE8 data-size study. Each run uses 16k accepted groups. Curves use 32-collection trailing averages, weighted by the number of attempted trajectories and including trajectories from rejected groups.
Figure 13: Full-test accuracy for the runs in Figure 12 . For each training seed, smaller question subsets are contained in larger ones.
Figure 14: Math comparison with concurrent TIPS. Solid blue denotes separate policies trained at each budget; the dashed and dash-dot ReMaRL curves reuse policies trained at 32 and 64 calls, respectively, across inference budgets. Our checkpoints use 32k groups. TIPS values are reported in its Table 4.
Figure 15: Code comparison with all four TIPS policy variants reported in Tables 24–25 of Zhang et al. (2026) . Solid blue denotes separate policies at their training NFE; dash-dot and dashed curves reuse the NFE16 and NFE32 checkpoints. Our policies use 32k groups.
Figure 16: Gains over the highest-scoring TIPS policy variant at each budget and benchmark. The reference can select a different variant at each point. At 64 calls, both of our fixed-checkpoint results are shown separately.
Run
Attempted groups
Accepted (%)
Math, NFE4
74,845
43.8
Math, NFE8
60,539
54.1
Math, NFE16
61,974
52.9
Math, NFE32
65,803
49.8
Math, NFE64
75,748
43.3
Code, NFE8
99,632
32.9
Appendix
Table 13: Question groups sampled to collect 32,768 accepted groups for each main policy.
Comparison (math, NFE8)
GSM8K
MATH500
Average
ReMaRL (32k) vs. P2 ′ , τ=1
+24.84 [22.87, 26.81]
+5.27 [3.00, 7.60]
+15.05 [13.55, 16.59]
ReMaRL (32k) vs. calibrated P2 ′
+11.30 [9.48, 13.12]
+1.13 [-0.93, 3.27]
+6.21 [4.88, 7.61]
ReMaRL (16k) vs. calibrated P2 ′
+7.73 [5.86, 9.60]
+2.13 [0.00, 4.33]
+4.93 [3.53, 6.35]
ReMaRL (32k) vs. remasking-matched P2 ′
+25.30 [23.35, 27.27]
+4.47 [2.33, 6.60]
+14.88 [13.44, 16.33]
ReMaRL (16k) vs. no remasking
+11.78 [9.98, 13.57]
+5.00 [2.80, 7.20]
+8.39 [6.96, 9.83]
ReMaRL (16k) vs. no hidden states
+4.95 [3.21, 6.72]
+1.80 [-0.07, 3.73]
+3.38 [2.08, 4.66]
Appendix
Table 14: Paired score differences in percentage points, with 95% bootstrap intervals in brackets. Average gives equal weight to the two benchmarks within each domain.