Designing regulatory DNA sequences for targeted gene-expression control is an important challenge. Recent approaches frame this task as property-conditioned generation, where diffusion models trained on natural sequences are guided toward high predicted activity through reward-based fine-tuning or sampling-time guidance. We show that single-reward evaluation can obscure reward-hacking behavior. By decomposing evaluation into property, novelty, and plausibility, we identify an empirically underexplored region of the evaluation space in which all three objectives are satisfied, but no existing baseline succeeds. We postulate that this gap arises because prior methods treat guidance primarily as a matter of magnitude. In regulatory DNA, however, sequence grammar is spatially nonuniform: motif positions are sensitive to perturbation, while background regions allow more exploration. We therefore propose LOCOGEN by reformulating guidance as a problem of locality. LOCOGEN learns an invariant motif mask through sequence-level invariance learning to protect motif positions, while applying OOD damping only to background positions. It operates at sampling time on a frozen diffusion backbone, without retraining the backbone. Across HepG2, K562, and SK-N-SH, LOCOGEN reaches this previously underexplored region of the property-novelty-plausibility space. Code is available at https://github.com/Lotfollahi-lab/LocoGen.
Figures & tables
Figure 1: LocoGen balances property, novelty, and plausibility.
Figure 2: Diagnostics of existing sequence design baselines.
Property
Novelty
Plausibility
Method
Pred-activity ↑
ATAC_acc ↑
Authenticity ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
Train
2.406
0.244
0.728
-
-
-
MDLM
0.157 ± 0.024
0.329 ± 0.198
0.777 ± 0.033
−0.15± 0.03
0.051 ± 0.006
0.071 ± 0.001
MDLM+CG
4.516 ± 0.151
0.481 ± 0.022
0.809 ± 0.014
+0.21± 0.02
0.169 ± 0.013
0.043 ± 0.007
DNA-Diff
2.595 ± 0.087
0.688 ± 0.293
0.821 ± 0.015
−0.07± 0.03
0.002 ± 0.002
0.010 ± 0.002
DRAKES
5.610 ± 0.070
0.804 ± 0.159
0.873 ± 0.004
+1.08± 0.09
0.261 ± 0.007
0.060 ± 0.010
Table 1: Three axis benchmark in HepG2 cell line. 1st , 2nd , and 3rd best results are highlighted.
Component
Property
Novelty
Plausibility
CG
Glob. OOD
Loc. OOD
Pred-activity ↑
ATAC_acc ↑
Authenticity ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
–
0.157 ± 0.024
0.329 ± 0.198
0.777 ± 0.033
−0.15± 0.03
0.051 ± 0.006
0.071 ± 0.001
✓
–
0.300 ± 0.028
0.314 ± 0.008
0.946 ± 0.002
+0.46± 0.03
0.246 ± 0.002
0.167 ± 0.007
Learnt
0.309 ± 0.020
0.311 ± 0.009
0.947 ± 0.003
+0.46± 0.03
0.244 ± 0.004
0.166 ± 0.006
✓
–
4.516 ± 0.151
0.481 ± 0.022
0.809 ± 0.014
+0.21± 0.02
0.169 ± 0.013
0.043 ± 0.007
✓
✓
–
4.608 ± 0.061
0.691 ± 0.041
0.887 ± 0.012
+0.62± 0.11
0.065 ± 0.021
0.044 ± 0.015
Table 2: Locality ablation on HepG2 cell line.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
Sequences and diffusion (§ 3 )
A
Nucleotide alphabet {A,T,G,C}
K=4
Alphabet size
L
DNA sequence length ( L=200 in all experiments)
ℓ,k
Position index and nucleotide-channel index
n,B
Sample index within a minibatch and minibatch size
Appendix
Table S1 : Summary of notation. Symbols are grouped by the section in which they are introduced.
Symbol
Value
Description
Source
H
128
hidden dim of CNN backbones
model.hidden_dim
stacks
5
dilated-CNN stack depth
model.num_cnn_stacks
dropout
0.1
per-conv dropout
model.dropout
ρ
0.3
target invariant mask density
init_rho
λinv
1.0
V-REx weight
config
λreg
5.0
mask-anchoring weight
config
Appendix
Table S2 : Hyperparameters for the masked separator (§ S1.3 ).
Symbol
Value
Description
γ
0.7
OOD damping strength ( dℓ,t=1−γ , Eq. 9 )
τM
0.5
mask threshold; damping acts where Mt,ℓ<τM
τtime
0.5
damping time threshold (active only for t<τtime )
c
3.0
target property-pull-to-prior magnitude ratio (Eq. 10 )
ηmax
20
cap on ηt when ∣∇r∣ is small
sentinel cutoff
−105
threshold below which logpθ is excluded from the norm
Appendix
Table S3 : Hyperparameters for the localization-guided sampler (§ S1.4 ).
Property
Novelty
Plausibility
Method
Pred-activity ↑
ATAC_acc ↑
Authenticity ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
Train
2.700
0.281
0.717
-
-
-
MDLM
0.106 ± 0.028
0.367 ± 0.227
0.777 ± 0.033
−0.03± 0.03
0.049 ± 0.005
0.067 ± 0.001
MDLM+CG
6.433 ± 0.052
0.852 ± 0.196
0.810 ± 0.006
+0.41± 0.03
0.260 ± 0.007
0.093 ± 0.005
DNA-Diff
2.979 ± 0.046
0.479 ± 0.311
0.841 ± 0.007
+0.13± 0.03
0.028 ± 0.020
0.038 ± 0.008
DRAKES
6.493 ± 0.155
0.863 ± 0.140
0.873 ± 0.004
+1.07± 0.08
0.251 ± 0.008
0.069 ± 0.011
Appendix
Table S4 : Three axis benchmark in K562 cell line ( n=640×3 seeds, mean ± std). 1st , 2nd , and 3rd best results are highlighted.
Property
Novelty
Plausibility
Method
Pred-activity ↑
ATAC_acc ↑
Authenticity ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
Train
2.376
0.212
0.713
-
-
-
MDLM
0.112 ± 0.026
0.391 ± 0.217
0.777 ± 0.033
−0.09± 0.03
0.071 ± 0.006
0.080 ± 0.001
MDLM+CG
5.047 ± 0.101
0.429 ± 0.295
0.828 ± 0.010
+0.23± 0.06
0.120 ± 0.003
0.007 ± 0.005
DNA-Diff †
0.033 ± 0.015
0.079 ± 0.087
0.821 ± 0.015
+0.09± 0.03
0.024 ± 0.000
0.034 ± 0.000
DRAKES
6.992 ± 0.169
0.879 ± 0.145
0.873 ± 0.004
+1.07± 0.09
0.241 ± 0.008
0.095 ± 0.011
Appendix
Table S5 : Three axis benchmark in SK-N-SH cell line ( n=640×3 seeds, mean ± std). 1st , 2nd , and 3rd best results are highlighted.
Figure S2 : The learnt invariant region is not a random subset of positions. (A) For ten HepG2 enhancers, the learnt invariant mask (dark) and a count-matched random mask (purple) along the 200bp window; the light band is the OOD-damped background. Both masks select the same number of positions, so any difference in the ablation is attributable to which positions are spared rather than how many. (B) Per-sequence Jaccard index between the two masks.
γ
Pred-activity↑
ATAC acc↑
Auth. ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
0.0
4.516±0.151
0.481±0.022
0.809±0.011
+0.22±0.01
0.169±0.010
0.043±0.005
0.3
4.738±0.061
0.651±0.007
0.870±0.004
+0.53±0.03
0.095±0.003
0.027±0.002
0.7
4.679±0.0
0.716±0.019
0.906±0.005
+0.71±0.00
0.055±0.004
0.057±0.002
Appendix
Table S6 : OOD strength sweep on HepG2.
Segmentation
Replacement
Property
Novelty
Plausibility
Pred-act. ↑
ATAC acc ↑
Authent. ↑
Rarity (IQR) ↑
∣ΔGini∣↓
∣Δhits∣↓
JASPAR PSSM
Uniform random
0.275
0.206
0.928
+0.25
0.155
0.114
JASPAR PSSM
Dinuc.-preserving
0.122
0.092
0.815
−0.12
0.069
0.066
GC content
Uniform random
0.359
0.252
0.907
+0.13
0.148
0.129
GC content
Dinuc.-preserving
0.078
0.086
0.809
−0.16
0.054
0.076
Saliency
Uniform random
0.288
0.249
0.961
+0.30
0.167
0.138
Appendix
Table S7 : Comparison with non-generative segment-and-shuffle controls on HepG2.
Activity
Accessibility
Method
DRAKES Pred-activity
SVDD DNA_evaluation alternative training split
DRAKES ATAC_acc
human-atac-catlas different data source
MDLM
0.157 (6)
0.408±0.013 (6)
0.329 (6)
0.050±0.009 (6)
MDLM+CG
4.516 (4)
4.434±0.097 (4)
0.481 (4)
0.459±0.014 (5)
DNA-Diff
2.595 (5)
2.721±0.046 (5)
0.688 (3)
0.517±0.020 (4)
DRAKES
5.610 (2)
5.135±0.076 (2)
0.804 (1)
0.829±0.006 (1)
SVDD
6.830 (1)
6.187±0.018 (1)
0.442 (5)
0.547±0.010 (3)
Appendix
Table S8: Robustness to oracle choice (HepG2). Beyond the DRAKES evaluation oracles used in the main text, we add two predictors that differ in training split and in data source, respectively. Rank across the six methods in parentheses; values are mean ± s.d. where seed replicates are available.
Activity
Accessibility
Method
DRAKES Pred-activity
SVDD DNA_evaluation alternative training split
DRAKES ATAC_acc
human-atac-catlas different data source
MDLM
0.106 (6)
0.381±0.031 (6)
0.367 (6)
0.055±0.006 (6)
MDLM+CG
6.433 (3)
6.191±0.042 (2)
0.852 (4)
0.688±0.022 (4)
DNA-Diff
2.979 (5)
3.038±0.036 (5)
0.479 (5)
0.359±0.009 (5)
DRAKES
6.493 (2)
6.031±0.114 (4)
0.863 (3)
0.837±0.005 (3)
SVDD
9.479 (1)
8.490±0.064 (1)
0.928 (1)
0.899±0.005 (1)
Appendix
Table S9: Robustness to oracle choice (K562). Same four oracles and conventions as Table S8 ; rank across the six methods in parentheses, values are mean ± s.d. where seed replicates are available.
Activity
Accessibility
Method
DRAKES Pred-activity
SVDD DNA_evaluation alternative training split
DRAKES ATAC_acc
human-atac-catlas different data source
MDLM
0.112 (5)
0.340±0.033 (6)
0.391 (4)
0.030±0.005 (6)
MDLM+CG
5.047 (4)
4.970±0.104 (4)
0.429 (3)
0.377±0.020 (3)
DNA-Diff
0.033 (6)
0.720±0.021 (5)
0.079 (6)
0.040±0.002 (5)
DRAKES
6.992 (2)
6.132±0.153 (2)
0.879 (1)
0.839±0.007 (1)
SVDD
7.989 (1)
6.714±0.103 (1)
0.356 (5)
0.192±0.004 (4)
Appendix
Table S10: Robustness to oracle choice (SK-N-SH). Same four oracles and conventions as Table S8 ; rank across the six methods in parentheses, values are mean ± s.d. where seed replicates are available.
Figure S3 : (A). Balanced score over three evaluation axis where LocoGen achieves the highest balanced score across sweep points. (B) Evaluation metrics across the sweep, with one row per hyperparameter setting. The value in parentheses indicates each method’s strength setting: γ denotes OOD/calibration strength ( ood_lambda ); r denotes the property-pull ratio ( alpha_ratio ) at γ=0.7 (the paper default r=3.0 corresponds to the γ=0.7 row, while r=1,2 are lower-pull variants); cap denotes the MDLM+CG classifier-guidance α -cap ( alpha_cap ); M denotes the number of SVDD candidates per step (Tweedie/SVDD-PM); αTDS denotes the SVDD-TDS importance-resampling temperature; and gs denotes the DNA-Diff classifier-free guidance scale ( cond_weight ; author default 1.0 ). DRAKES fine-tunes model weights for a fixed reward rather than steering generation at inference time, and is therefore included as a retraining reference rather than a directly comparable operating point.
We present a parameter-efficient Diffusion Transformer (DiT) for generating 200bp cell-type-specific regulatory DNA sequences. By replacing the U-Net backbone of DNA-Diffusion with a transformer denoiser equipped with a 2D CNN input encoder, our model matches the U-Net's best validation loss in 13 epochs (60× fewer) and converges 39% lower, while reducing memorization from 5.3% to 1.7% of generated sequences aligning to training data via BLAT. Ablations show the CNN encoder is essential: without it, validation loss increases 70% regardless of positional embedding choice. We further apply DDPO finetuning using Enformer as a reward model, achieving a 38× improvement in predicted regulatory activity. Cross-validation against DRAKES on an independent prediction task confirms that improvements reflect genuine regulatory signal rather than reward model overfitting.
Jonathan Liu, Kia Ghods
Department of Computer Science Princeton University
We study the application of recent Edit Flows for inference-time reward control for DNA sequence generation. Unlike most reward-guided DNA generation frameworks, which operate on fixed-length sequence spaces, Edit Flows have a potential to generate variable-length DNA through biologically plausible insertion, deletion, and substitution operations. In particular, we propose Local Perturbation Discrete Programming (LPDP), a training-free, intermediate-state and action-aware local re-solving operator for variable-length DNA edit-action generators at inference time. More specifically, at each guided rollout step, LPDP scores one-step root edits, retains a near-best root band, and re-ranks each retained root by solving a bounded local discrete program around its child sequence. This local program uses the typed geometry of edit actions to focus on coherent substitution, insertion, or deletion subgraphs, and aggregates local continuations with either a hard Max backup or a soft log-sum-exponential (LSE) backup. We instantiate LPDP in two regimes: front-loaded reward tilting for enhancer optimization, where early edits are critical for establishing global regulatory sequence structure, and back-loaded reward tilting for exon-intron-exon inpainting, where late edits fine-tune splice-boundary contexts.
Early DNA foundation models adopted BERT-style training, achieving good performance on DNA understanding tasks but lacking generative capabilities. Recent autoregressive models enable DNA generation, but employ left-to-right causal modeling that is suboptimal for DNA where regulatory relationships are inherently bidirectional. We present D3LM (Discrete DNA Diffusion Language Model), which unifies bidirectional representation learning and DNA generation through masked diffusion. D3LM directly adopts the Nucleotide Transformer (NT) v2 architecture but reformulates the training objective as masked diffusion in discrete DNA space, enabling both bidirectional understanding and generation capabilities within a single model. Compared to NT v2 of the same size, D3LM achieves improved performance on understanding tasks. Notably, on regulatory element generation, D3LM achieves an SFID of 10.92, closely approaching real DNA sequences (7.85) and substantially outperforming the previous best result of 29.16 from autoregressive models. Our work suggests diffusion language models as a promising paradigm for unified DNA foundation models. We further present the first systematic study of masked diffusion models in the DNA domain, investigating practical design choices such as tokenization schemes and sampling strategies, thereby providing empirical insights and a solid foundation for future research. D3LM has been released at https://huggingface.co/collections/Hengchang-Liu/d3lm.
Zhao Yang, Hengchang Liu, Chuan Cao +1
Gaoling School of Artificial Intelligence, Renmin University of China · Zhongguancun Academy