Slithering Through Gaps: Capturing Discrete Isolated Modes via Logistic Bridging

Pinaki Mohanty; Ruqi Zhang

cs.LGApr 12, 2026

Slithering Through Gaps: Capturing Discrete Isolated Modes via Logistic Bridging

Authors: Pinaki Mohanty, Ruqi Zhang

Organizations: Department of Computer Science, College of Science & College of Engineering, Purdue University, West Lafayette, IN, USA

Abstract

High-dimensional and complex discrete distributions often exhibit multimodal behavior due to inherent discontinuities, posing significant challenges for sampling. Gradient-based discrete samplers, while effective, frequently become trapped in local modes when confronted with rugged or disconnected energy landscapes. This limits their ability to achieve adequate mixing and convergence in high-dimensional multimodal discrete spaces. To address these challenges, we propose Hyperbolic Secant-squared Gibbs-Sampling (HiSS), a novel family of sampling algorithms that integrates a Metropolis-within-Gibbs framework to enhance mixing efficiency. HiSS leverages a logistic convolution kernel to couple the discrete sampling variable with the continuous auxiliary variable in a joint distribution. This design allows the auxiliary variable to encapsulate the true target distribution while facilitating easy transitions between distant and disconnected modes. We provide theoretical guarantees of convergence and demonstrate empirically that HiSS outperforms many popular alternatives on a wide variety of tasks, including Ising models, binary neural networks, and combinatorial optimization.

Figures & tables

Figure 1: A visualization of the multimodal cost landscape in the Traveling Salesman Problem, showcasing solutions separated by low energy barriers.

Figure 2: 4D Joint Bernoulli

Figure 3: Comparison of the target distribution, tempered distribution, and

p(\bm{\theta}_{a})

in HiSS.

Figure 4: Ising Model

Figure 5: Gaussian Kernel vs Logistic Kernel

Panel of Figure 6 — Figure 6: MH vs no-MH

Figure 7: Sensitivity analysis of HiSS.

Figure 8: No Gradient Refinement for 4D Bernoulli

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Mode Bridging in Bernoulli Distribution

Figure 10: Intermediate Mass for Kernels

Figure 11: Target Distribution for 4D Joint Bernoulli

Figure 12: Coverage Analysis for 4D Bernoulli

Figure 14: Coverage Analysis for Ising Model

Figure 15: Impact of Gibbs Sweeps and Refinement Iterations in Ising models.

Sampler	# Energy Evaluations
GWG	$4.0\times 10^{5}$
DMALA	$4.0\times 10^{5}$
ACS	$4.0\times 10^{5}$
PT+DMALA	${2.2\times 10^{6}}$
HiSS	${5.0\times 10^{5}}$

Appendix

Table 3: Number of Energy Evaluations in 4D Joint Bernoulli

Sampler	# Energy Evaluations
GWG	$1.00\times 10^{6}$
DMALA	$1.00\times 10^{6}$
ACS	$1.00\times 10^{6}$
PT+DMALA	${6.00\times 10^{6}}$
HiSS	${1.25\times 10^{6}}$

Appendix

Table 4: Number of Energy Evaluations in Ising Models

Panel of Figure 17 — Figure 17: Impact of scale of logistic noise on solution quality.

Dataset	Average Training Log-likelihood ( $\uparrow$ )					Average Training Root-Mean Square Error ( $\downarrow$ )
Dataset	GWG	DMALA	ACS	PT+DMALA	HiSS	GWG	DMALA	ACS	PT+DMALA	HiSS
Breast Cancer	-0.0708 ±0.0089	-0.0693 ±0.0025	-0.0938 ±0.0060	-0.0721 ±0.0049	-0.0728 ±0.0028	0.0283 ±0.0032	0.0276 ±0.0011	0.0315 ±0.0018	0.0287 ±0.002	0.0291 ±0.0013
COMPAS	-0.3130 ± 0.0068	-0.3121 ± 0.0027	-0.3139 ± 0.0037	-0.3149 ± 0.0057	-0.3697 ± 0.0073	0.2213 ±0.0019	0.2219 ±0.0020	0.2230 ±0.0020	0.2215 ± 0.0020	0.2172 ± 0.0024
HIV	-0.7453 ± 0.0750	-0.7746 ± 0.0000	-0.7746 ± 0.0000	-0.7746 ± 0.0000	-0.4578 ± 0.0101	0.7299 ± 0.1205	0.7746 ± 0.0000	0.7746 ± 0.0000	0.7746 ± 0.0000	0.2554 ± 0.0100
Blog	-0.3458 ± 0.0127	-0.3476 ± 0.0000	-0.3476 ± 0.0000	-0.3476 ± 0.0000	-0.4094 ± 0.0092	0.3320± 0.0453	0.3476 ± 0.0000	0.3476 ± 0.0000	0.3476 ± 0.0000	0.2043 ± 0.0088

Appendix

Table 5: Experiment results with binary Bayesian neural networks on different training datasets.

Explore similar work

Fixed-point neural samplers on discrete spaces
Oct 1, 2026Jiajun He, Denis Blessing, Mouyang Cheng +2Diffusion Models
MIT · Microsoft Research New England
Gradient-free Riemannian Langevin Sampler
Jul 8, 2026Ricardo Baptista, Olivier ZahmMarkov Chain Monte Carlo
Department of Statistical Sciences, University of Toronto, Canada · UGA, Inria, CNRS, Grenoble INP*, LJK, 38000 Grenoble, France
A Generative Sampler for distributions with possible discrete parameter based on Reversibility
Mar 10, 2026Lei Li, Zhen Wang, Lishuo ZhangGenerative Modeling Markov Chain Monte Carlo
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, 200240, P.R.China. · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, 200240, P.R.China.

Oct 1, 2026J/K move · Enter open · S save

Fixed-point neural samplers on discrete spaces

Jiajun He, Denis Blessing, Mouyang Cheng, Yuanqi Du+1

MIT · Microsoft Research New England

Diffusion Models

Open paper AskPDF arXiv

Sampling from discrete, unnormalized distributions without access to data is a challenging problem. Neural samplers offer a promising approach by training generative models from density evaluations directly. Despite recent progress, existing discrete neural samplers are prone to mode collapse, come without convergence guarantees when trained via fixed-point iterations, and are often tied to a specific reference process such as masked or uniform diffusion. In this work, we introduce Discrete Gibbs Iterative Neural Sampler, a fixed-point neural sampler that addresses these limitations, enabling efficient, scalable learning, substantially reducing mode collapse in practice. Our framework builds on masked diffusion and also extends to transport between pairs of distributions. We demonstrate that the resulting method scales effectively to high-dimensional systems, supports amortized sampling across different conditions, and enables accurate estimation of alloy phase diagrams.

Abstract

Explore similar work

Fixed-point neural samplers on discrete spaces

Gradient-free Riemannian Langevin Sampler

A Generative Sampler for distributions with possible discrete parameter based on Reversibility