PassGPT+: Leveraging Linguistic Priors for Password Modeling

Rajneesh Anand; Neeraj Lakshmanan; Masoud Yari

cs.CRSep 30, 2026

PassGPT+: Leveraging Linguistic Priors for Password Modeling

Authors: Rajneesh Anand, Neeraj Lakshmanan, Masoud Yari

Organizations: Department of Chemical and Biomolecular Engineering, Lehigh University, Bethlehem, PA, 18015, USA · Department of Computer Science & Design, Singapore University of Technology & Design, Singapore · Department of Computer Science & Engineering, Lehigh University, Bethlehem, PA, 18015, USA

Abstract

Passwords remain the dominant online authentication mechanism, and understanding how humans choose them is essential for defensive strength estimation and attack simulation alike. Recent learning-based approaches such as PassGAN and PassGPT have shown that deep generative models can learn password structure directly from leaked corpora. However, both train from random initialization on password data alone. The role of linguistic prior knowledge in password modeling, and what it reveals about how humans create secrets, remains largely underexplored. Here, we address this gap with PassGPT+, which adapts the linguistic prior of GPT-2 to password observations through character-aware tokenization. We also introduce PassDiffusion, the first absorbing-state discrete diffusion model for password generation, as a probe of whether non-autoregressive approaches are competitive. On the RockYou benchmark, PassGPT+ recovers 22.53% of held-out passwords at 108 guesses, a 16% relative gain over PassGPT, and retains 79% of this match rate when transferred without retraining to a disjoint 2020 leak dataset, demonstrating that linguistic priors capture persistent regularities of human password generation. PassDiffusion underperforms by two to three orders of magnitude, indicating that autoregressive modeling is substantially better matched than iterative denoising to the discrete, exact-match nature of password generation.

Figures & tables

Figure 1: Proposed architectures for password generation. (a) PassGPT+: a 12-layer causal Transformer decoder that adapts a variant of GPT-2 architecture and is trained using its weights through character-level tokenization. Input characters are mapped to GPT-2 BPE token IDs and processed through 12 masked self-attention blocks (12 heads, head dim 64) with feed-forward expansion ( $4\times$ hidden dim), layer normalization, and residual connections. Generation is seeded with a random in-distribution character rather than the standard BOS token, avoiding out-of-distribution initialization. (b) PassDiffusion (or PassDiff): an absorbing-state discrete diffusion model for password generation. A real password $x_{0}$ is corrupted via forward masking $q(x_{t}\mid x_{0})$ over $T=1000$ timesteps, producing a progressively masked sequence $x_{t}$ . A bidirectional Transformer encoder (6 layers, 8 heads, hidden dim 256) conditioned on timestep $t$ learns to predict $x_{0}$ from $x_{t}$ . At inference, passwords are generated by iterative reverse unmasking over 200 steps from a fully masked sequence.

Aspect	PassGPT [ 20 ]	PassGPT+ (Ours)
Initialization	Uninformative (random init)	GPT-2 linguistic prior
Tokenizer	Custom 256-char	GPT-2 BPE (char-mapped)
Layers	8 layers, 12 heads	12 layers, 12 heads
Parameters	${\sim}85$ M	117M
Training epochs	1	3
Generation seed	BOS ( <s> )	Random password char

Table 1: Comparison between the model architecture of PassGPT [ 20 ] and our PassGPT+. The three changes most relevant to performance are linguistic prior initialization, increased depth, and additional fine-tuning epochs.

Guess	PassGAN	PassGAN*	PassVQT	PassGPT	PassDiffusion	HashCat	PassGPT+
	[ 9 ]	[ 16 ]	[ 20 ]	[ 20 ]	(Ours)		(Ours)
$10^{4}$	.010	—	.0040	.010	.0000	.0095	.0093
$10^{5}$	.050	—	.050	.050	.0002	.092	.085
$10^{6}$	.380	—	.450	.500	.0018	.690	.780
$10^{7}$	2.04	—	2.90	4.25	.030	4.65	5.66
$10^{8}$	6.73	9.51	10.30	19.37	—	—	22.53

Table 2: Match rate (%) on the RockYou test set across generation sizes. Best result per row is highlighted in yellow. Em-dashes (—) denote values not reported in the original publications. PassGPT+ wins at every generation size of

10^{6}

or larger.

Figure 2: Comparison of PassGPT+ against all other models on the RockYou test set. (a) Match rate of each model on a logarithmic vertical axis. PassGPT+ (red, outlined in black) achieves the highest match rate at every generation size from $10^{6}$ guesses onward. The very low bars for PassDiffusion confirm that the discrete diffusion framework is poorly matched to exact-character password generation. (b) Relative improvement of PassGPT+ over each learning-based baseline and Hashcat, computed as $[(\text{PassGPT+}-\text{other model})/\text{other model}]\times 100\%$ . Positive values indicate that PassGPT+ recovers more passwords than the baseline at that generation size. The gain over PassGPT (the closest competitor) grows from $+56\%$ at $10^{6}$ to $+16\%$ at $10^{8}$ , while the gain over PassGAN exceeds $+200\%$ at the largest generation size. Hashcat marginally wins at very small generation sizes ( $10^{4}$ and $10^{5}$ ), reflecting the well-known efficiency of hand-crafted rules at low guess counts, but PassGPT+ overtakes it from $10^{6}$ onward.

Figure 3: Cross-distribution robustness of PassGPT+ evaluated on a 2020 leaked password corpus that excludes any password appearing in the RockYou training set. (Top) Match rates on the RockYou test set (blue) and the unseen 2020 set (orange). Although absolute performance decreases on the newer distribution as expected for any model trained on a single historical leak, the drop is modest and PassGPT+ still recovers $17.72\%$ of the 2020 test passwords at $10^{8}$ guesses. (Bottom) Performance retention, defined as the ratio of the 2020 match rate to the RockYou match rate at the same generation size, expressed as a percentage. Retention is consistently above the $50\%$ baseline (dashed gray line) and increases with the generation size.

Explore similar work

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
Jun 7, 2026Antonio Franca, Alexander TongLanguage Model Perplexity Single-Token Output Distributions
AITHYRA Institute, Vienna, Austria.
Gacha Decoding: Eliciting Diverse Generations Through Instruction Following
Oct 1, 2026Scott Geng, Yufei Zhang, Joseph Lee +3Large Language Model Generation Diversity
University of Washington · Meta Superintelligence Labs
Chainwash: Multi-Step Rewriting Attacks on Diffusion Language Model Watermarks
May 6, 2026Mohd Ruhul Ameen, Akif Islam, Nadim Mahmud +1Large Language Model Watermarks Watermarks
College of Engineering and Computer Science, Marshall University, Huntington, WV, USA · Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh · Miami University, Oxford, OH 45056, USA

Jun 7, 2026, cs.CLJ/K move · Enter open · S save

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics

Antonio Franca, Alexander Tong

AITHYRA Institute, Vienna, Austria.

Language Model Perplexity Single-Token Output Distributions Autoregressive Model

Open paper AskPDF arXiv

Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling. Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer such as gpt2-large, typically paired with an empirical-entropy guardrail to rule out low-entropy collapse. We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence -- and the set of predictable but still low-quality sequences is combinatorially large. To make this concrete, we construct a suite of zero-parameter, deliberately naive samplers that achieve state-of-the-art gen-PPL on LM1B and OpenWebText at non-degenerate entropy, surpassing recently published diffusion and continuous-flow models while producing text that is incoherent by construction. We recommend evaluation suites that directly quantify the distributional divergence between generated and reference text, and use such a suite to re-benchmark recent non-autoregressive models, recovering a more faithful picture of the current state of the art.

Abstract

Explore similar work

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics

Gacha Decoding: Eliciting Diverse Generations Through Instruction Following

Chainwash: Multi-Step Rewriting Attacks on Diffusion Language Model Watermarks