PassGPT+: Leveraging Linguistic Priors for Password Modeling

Rajneesh Anand; Neeraj Lakshmanan; Masoud Yari

cs.CRSep 30, 2026

PassGPT+: Leveraging Linguistic Priors for Password Modeling

Authors: Rajneesh Anand, Neeraj Lakshmanan, Masoud Yari

Organizations: Department of Chemical and Biomolecular Engineering, Lehigh University, Bethlehem, PA, 18015, USA · Department of Computer Science & Design, Singapore University of Technology & Design, Singapore · Department of Computer Science & Engineering, Lehigh University, Bethlehem, PA, 18015, USA

Abstract

Passwords remain the dominant online authentication mechanism, and understanding how humans choose them is essential for defensive strength estimation and attack simulation alike. Recent learning-based approaches such as PassGAN and PassGPT have shown that deep generative models can learn password structure directly from leaked corpora. However, both train from random initialization on password data alone. The role of linguistic prior knowledge in password modeling, and what it reveals about how humans create secrets, remains largely underexplored. Here, we address this gap with PassGPT+, which adapts the linguistic prior of GPT-2 to password observations through character-aware tokenization. We also introduce PassDiffusion, the first absorbing-state discrete diffusion model for password generation, as a probe of whether non-autoregressive approaches are competitive. On the RockYou benchmark, PassGPT+ recovers 22.53% of held-out passwords at 108 guesses, a 16% relative gain over PassGPT, and retains 79% of this match rate when transferred without retraining to a disjoint 2020 leak dataset, demonstrating that linguistic priors capture persistent regularities of human password generation. PassDiffusion underperforms by two to three orders of magnitude, indicating that autoregressive modeling is substantially better matched than iterative denoising to the discrete, exact-match nature of password generation.

Figures & tables

Figure 1: Proposed architectures for password generation. (a) PassGPT+: a 12-layer causal Transformer decoder that adapts a variant of GPT-2 architecture and is trained using its weights through character-level tokenization. Input characters are mapped to GPT-2 BPE token IDs and processed through 12 masked self-attention blocks (12 heads, head dim 64) with feed-forward expansion ( $4\times$ hidden dim), layer normalization, and residual connections. Generation is seeded with a random in-distribution character rather than the standard BOS token, avoiding out-of-distribution initialization. (b) PassDiffusion (or PassDiff): an absorbing-state discrete diffusion model for password generation. A real password $x_{0}$ is corrupted via forward masking $q(x_{t}\mid x_{0})$ over $T=1000$ timesteps, producing a progressively masked sequence $x_{t}$ . A bidirectional Transformer encoder (6 layers, 8 heads, hidden dim 256) conditioned on timestep $t$ learns to predict $x_{0}$ from $x_{t}$ . At inference, passwords are generated by iterative reverse unmasking over 200 steps from a fully masked sequence.

Aspect	PassGPT [ 20 ]	PassGPT+ (Ours)
Initialization	Uninformative (random init)	GPT-2 linguistic prior
Tokenizer	Custom 256-char	GPT-2 BPE (char-mapped)
Layers	8 layers, 12 heads	12 layers, 12 heads
Parameters	${\sim}85$ M	117M
Training epochs	1	3
Generation seed	BOS ( <s> )	Random password char

Table 1: Comparison between the model architecture of PassGPT [ 20 ] and our PassGPT+. The three changes most relevant to performance are linguistic prior initialization, increased depth, and additional fine-tuning epochs.

Guess	PassGAN	PassGAN*	PassVQT	PassGPT	PassDiffusion	HashCat	PassGPT+
	[ 9 ]	[ 16 ]	[ 20 ]	[ 20 ]	(Ours)		(Ours)
$10^{4}$	.010	—	.0040	.010	.0000	.0095	.0093
$10^{5}$	.050	—	.050	.050	.0002	.092	.085
$10^{6}$	.380	—	.450	.500	.0018	.690	.780
$10^{7}$	2.04	—	2.90	4.25	.030	4.65	5.66
$10^{8}$	6.73	9.51	10.30	19.37	—	—	22.53

Table 2: Match rate (%) on the RockYou test set across generation sizes. Best result per row is highlighted in yellow. Em-dashes (—) denote values not reported in the original publications. PassGPT+ wins at every generation size of

10^{6}

or larger.

Figure 2: Comparison of PassGPT+ against all other models on the RockYou test set. (a) Match rate of each model on a logarithmic vertical axis. PassGPT+ (red, outlined in black) achieves the highest match rate at every generation size from $10^{6}$ guesses onward. The very low bars for PassDiffusion confirm that the discrete diffusion framework is poorly matched to exact-character password generation. (b) Relative improvement of PassGPT+ over each learning-based baseline and Hashcat, computed as $[(\text{PassGPT+}-\text{other model})/\text{other model}]\times 100\%$ . Positive values indicate that PassGPT+ recovers more passwords than the baseline at that generation size. The gain over PassGPT (the closest competitor) grows from $+56\%$ at $10^{6}$ to $+16\%$ at $10^{8}$ , while the gain over PassGAN exceeds $+200\%$ at the largest generation size. Hashcat marginally wins at very small generation sizes ( $10^{4}$ and $10^{5}$ ), reflecting the well-known efficiency of hand-crafted rules at low guess counts, but PassGPT+ overtakes it from $10^{6}$ onward.

Figure 3: Cross-distribution robustness of PassGPT+ evaluated on a 2020 leaked password corpus that excludes any password appearing in the RockYou training set. (Top) Match rates on the RockYou test set (blue) and the unseen 2020 set (orange). Although absolute performance decreases on the newer distribution as expected for any model trained on a single historical leak, the drop is modest and PassGPT+ still recovers $17.72\%$ of the 2020 test passwords at $10^{8}$ guesses. (Bottom) Performance retention, defined as the ratio of the 2020 match rate to the RockYou match rate at the same generation size, expressed as a percentage. Retention is consistently above the $50\%$ baseline (dashed gray line) and increases with the generation size.

Explore similar work

Jun 7, 2026cs.CL

Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics

Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling. Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer such as gpt2-large, typically paired with an empirical-entropy guardrail to rule out low-entropy collapse. We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence -- and the set of predictable but still low-quality sequences is combinatorially large. To make this concrete, we construct a suite of zero-parameter, deliberately naive samplers that achieve state-of-the-art gen-PPL on LM1B and OpenWebText at non-degenerate entropy, surpassing recently published diffusion and continuous-flow models while producing text that is incoherent by construction. We recommend evaluation suites that directly quantify the distributional divergence between generated and reference text, and use such a suite to re-benchmark recent non-autoregressive models, recovering a more faithful picture of the current state of the art.

Antonio Franca, Alexander Tong

AITHYRA Institute, Vienna, Austria.

Language Model Perplexity Single-Token Output Distributions Autoregressive Model

Ask PDF

Oct 1, 2026cs.AI

Gacha Decoding: Eliciting Diverse Generations Through Instruction Following

We introduce Gacha Decoding, an inference-time method for eliciting diverse language model generations that scales with model capability. Across open-ended domains (in-the-wild chat, creative writing, planning for image generation, and protein design), Gacha Decoding significantly outperforms existing generation diversity approaches at equal quality (up to 2.4x Vendi over the next-best prior approach), reaching the same number of high-quality modes with over an order of magnitude fewer samples (11.0x) and discovering novel modes that no other approach surfaces. Our key insight is to treat diversity as an instruction-following problem: rather than relying on the LM's token entropy, we combine its instruction-following capability with randomness from an external RNG tool to scalably identify and realize distinct modes of the response space. This approach of "planning with dice" enables Gacha to invert the long-observed tension between diversity and model capability. As the underlying LM becomes a better instruction follower, diversity under Gacha Decoding consistently improves--even as its token entropy and diversity under prior approaches decline. Together, our results highlight that instruction following, rather than token entropy alone, can drive generation diversity.

Scott Geng, Yufei Zhang, Joseph Lee +3

University of Washington · Meta Superintelligence Labs

Large Language Model Generation Diversity Decoding Instruction

Ask PDF

May 6, 2026cs.CL

Chainwash: Multi-Step Rewriting Attacks on Diffusion Language Model Watermarks

Statistical watermarking is a common approach for verifying whether text was written by a language model. Most existing schemes assume autoregressive generation, where tokens are produced left to right and contextual hashing is well defined. Diffusion language models generate text by denoising tokens in arbitrary order, so these schemes cannot be applied directly. A recent watermark by Gloaguen et al. addresses this gap for LLaDA 8B Instruct and reports true positive detection above 99%. This paper studies what happens when watermarked text is rewritten not once but several times. Using the same watermark configuration, 1,605 watermarked completions of about 300 tokens each are produced across five WaterBench domains. Each completion is rewritten by four open weight language models, from 1.5B to 8B parameters, none of which know the watermark key. Five rewrite styles are tested: paraphrase, humanize, simplify, academic, and summarize expand. Each style is chained for up to five hops, producing 160,500 rewritten texts in total. The watermark is detected on 87.9% of the original outputs at the standard significance threshold. After a single rewrite, detection falls to between 14% and 41% depending on the rewriter and style. After five chained rewrites, detection falls to 4.86%, meaning 94.76% of the originally detected texts are no longer flagged. After three rewrites, the detector score has dropped 86% of the way from its watermarked baseline toward the null distribution. Repeated rewriting is therefore a much stronger attack than a single rewrite, and the result holds across all four rewriters tested.

Mohd Ruhul Ameen, Akif Islam, Nadim Mahmud +1

College of Engineering and Computer Science, Marshall University, Huntington, WV, USA · Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh · Miami University, Oxford, OH 45056, USA

Large Language Model Watermarks Watermarks Diffusion Language Models Attacker Large Language Model Machine-Generated Text Detection Rewrite+1

Ask PDF