Restricted Boltzmann Machines

Also known as RBM

Ask about this topic

Overview

Restricted Boltzmann Machines study how a simple network of visible and hidden units can represent and learn probability distributions. Researchers train them with sampling-based methods such as Gibbs updates and deep tempering, while theoretical work examines mixing times, model capacity, and how choices like activation functions affect learning. Applications include generating data, rejecting out-of-distribution inputs, modeling audio, and representing quantum states or molecular configurations. A central challenge is that sampling can be slow or unstable, and proposed alternatives—including quantum samplers—must be compared fairly with well-tuned classical methods.

Latest papers

CardsList

18 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Oct 29, 2026

Going deep, reaching further: What can Boltzmann machines efficiently represent?

While universal approximation theorems theoretically underpin the representation capabilities of machine learning models, they have two shortcomings: They do not focus on efficient representations and they tend to be insensitive to depth and other heuristically important properties. In this work we show that these two shortcomings are intertwined. We prove that machine learning models which are deeper or have higher-order interactions can efficiently represent a larger set of functions, i.e. have larger efficient reach. It follows that deep Boltzmann machines (DBMs) have maximal efficient reach, restricted Boltzmann machines (RBMs) sub-maximal, and k-body generalizations of RBMs form a hierarchy between the two, even though all are universal. More generally, (i) we provide a full characterization of machine learning models with maximal efficient reach; (ii) we derive separations between deep and shallow models for three representation tasks; and (iii) we prove that the efficient reach of shallow models constitutes an infinite hierarchy indexed by the interaction order. The proofs draw on different methodologies, namely on (i) a recent characterization of universal spin models; (ii) computational complexity bounds for spin models; and (iii) communication complexity bounds applied to shallow models. This work shows that universal machine learning models can strongly differ in their efficient representation capabilities, and provides a theoretical foundation for heuristic observations.

Tobias Reinhart, Gemma De les Coves

Institute for Theoretical Physics, University of Innsbruck,, Technikerstr. 21a, A-6020 Innsbruck, Austria · ICREA, Institució Catalana d’Estudis i Recerca Avançats, Passeig Lluís Companys 23, 08010 Barcelona · Departament d’Enginyeria, Universitat Pompeu Fabra, Carrer Tànger 122, 08018 Barcelona

Restricted Boltzmann Machines
AskPDF
Oct 17, 2026

Recurrent Temporal Restricted Boltzmann Machines as Unsupervised Encoders for Biomechanical Movement Sequences

Biomechanical movement recordings are continuous, high-dimensional time series, and analyzing them without manually labeling them requires representations that capture how movements evolve over time. Most unsupervised representation-learning pipelines, however, treat each observation frame independently and discard this temporal structure, while energy-based models that could capture it have not extensively been adapted to biomechanical signals. So, this paper implements the Recurrent Temporal Restricted Boltzmann Machine (RTRBM), a recurrent energy-based model, into an unsupervised encoder that maps windows of movement data to fixed-length embeddings, and tests what its recurrent structure contributes. We adapt the model to continuous-valued signals, stack it into a deep architecture, and integrate it into the SIT-FUSE segmentation pipeline. We evaluate the embeddings on simulated musculoskeletal reaching and on two real gait datasets, using clustering, reconstruction, and walking-speed classification of unseen subjects, and compare them against non-recurrent and non-probabilistic baselines, including a control that removes the recurrent connection. Clustering the embeddings recovers movement structure tied to progress toward the target without any labels, which also transfers to unseen episodes. On gait data, the embeddings outperform PCA and autoencoder baselines under a nonlinear probe. However, removing the recurrent connection does not yield a measurable loss on the reaching task, and simple per-channel summary statistics classify walking speed at least as well as the embeddings. The model is therefore a working, verified encoder for SIT-FUSE, while the evidence for a benefit of recurrence is mixed, and the learned embeddings did not outperform simple summary statistics on gait.

Wafi Sharif, Nicholas LaHaye

Queens High School for the Sciences · Jet Propulsion Laboratory, California Institute of Technology

Motion Representation LearningTime Series Representation LearningRestricted Boltzmann MachinesGait Analysis
AskPDF
Oct 12, 2026

Feature-Context Consistency: Unsupervised Adversarial Detection with Drift Stability on Attributed Graphs

Adversarial detection on evolving attributed graphs faces two challenges: fraud labels arrive late or not at all, and natural distributional drift erodes any trained decision boundary. Modern detectors can reach strong detection rates when labels are available and the distribution is stationary; sustaining that quality without labels and under drift is the open problem. We identify feature-context consistency -- the alignment between a node's attributes and its aggregated neighborhood attributes -- as a single structural signal that addresses both. We formalize it as the Context Boost (CB) score within a Restricted Boltzmann Machine framework (CB-RBM) and prove two complementary worst-case guarantees: a strict separation between the expected CB of legitimate and adversarial nodes, with gap growing linearly in model capacity, whenever the adversary's context stays far enough from legitimate neighborhood context; and a Wasserstein-Lipschitz bound on how far the CB distribution moves under natural drift. We validate on XBlock Ethereum phishing (~153K nodes) and Reddit banned-user detection (~11K nodes), with two further datasets in the appendix spanning diverse domains. With zero labels, CB-RBM achieves AUROC >= 0.96 on four adversarial attacks and keeps FPR close to its calibrated 5% target under drift -- outperforming supervised GNNs (hundreds of labels), unsupervised graph-anomaly baselines, and a standard RBM whose FPR collapses to 89-100%. What typically requires separate mechanisms -- label-free operation and drift stability -- here follows from a single structural regularity of the data. Code: https://github.com/serval-uni-lu/CB-RBM.

Firas Bayram, Maxime Cordy

SnT, University of Luxembourg Luxembourg

Graph-Based Anomaly DetectionAnomaly DetectionUnsupervised Anomaly DetectionAdversarial Example DetectionConcept DriftRestricted Boltzmann Machines
AskPDF
Sep 22, 2026

Hyperbolic Restricted Boltzmann Machine Neural Quantum State

We construct the first type of non-Euclidean non-autoregressive neural quantum state (NQS) in the form of the hyperbolic Restricted Boltzmann Machine (HRBM), which is studied in the variational Monte-Carlo (VMC) setting of the Quantum Sherrington-Kirkpatrick (QSK) model whose ground state exhibits volume-law entanglement. Across a 512-fold increase in the Hilbert space dimension corresponding to a system size increase from N=14N=14 to N=24N=24, HRBM NQS robustly outperforms its Euclidean version, the RBM NQS, in terms of better ground state energy optimization as well as lower Renyi-2 S2S_2 and von Neumann SvNS_{vN} absolute entanglement entropy reconstruction errors. More importantly, for all tested QSK system sizes, HRBM NQS demonstrates a superior expressivity in faithfully reproducing the entire entanglement spectrum of the QSK model from the top eigenvalues down to the tail end across 15 orders of magnitude, while RBM NQS consistently overestimates the sub-dominant modes. This work furnishes a proof-of-concept demonstrating that hyperbolic non-autoregressive NQS ansatze, thanks to the exponential volume of the hyperbolic geometry underlying their constructions, might be more natural at representing volume-law quantum systems than conventional Euclidean NQS. Furthermore, an interesting byproduct of this work is the polynomial scaling result of RBM-type NQS ansatze in the QSK volume-law system as the Hilbert space dimension increases exponentially.

H. L. Dao

Neural Quantum StatesQuantum Many-Body PhysicsRestricted Boltzmann MachinesHyperbolic Representation Learning
AskPDF
Aug 18, 2026

Nonlocal Transition Kernel for Efficient Learning of Restricted Boltzmann Machines

Learning restricted Boltzmann machines (RBMs) is computationally challenging because it requires expectations whose exact evaluation is generally intractable. The expectations are typically evaluated using a sampling approximation based on blocked Gibbs sampling (BGS), which is a local Markov chain Monte Carlo transition kernel. However, the locality of BGS can lead to poor sampling quality when the RBM has high energy barriers, thereby degrading learning performance. Deep tempering (DT), which performs parallel tempering over a sequence of learnable RBMs including the training RBM, alleviates this locality issue. However, DT algorithmically requires multiple steps to move through the RBM sequence to achieve a nonlocal transition. In this paper, we propose a transition kernel defined over the RBM sequence used in DT. The proposed kernel has a round-trip structure over the sequence, enabling nonlocal moves within a single transition while leaving the RBM sequence invariant. Numerical experiments show that the proposed kernel performs nonlocal transitions more frequently and achieves higher sampling quality with fewer transitions than BGS and DT. We further verify that learning based on the proposed kernel is more stable and mitigates the training failures observed with BGS- and DT-based learning.

Kaiji Sekimoto, Muneki Yasuda

Graduate School of Science and Engineering, Yamagata University, Japan

Restricted Boltzmann MachinesMarkov Chain Monte CarloParallel Tempering
AskPDF
Jul 27, 2026

Variational Quantum Conditional Boltzmann Machines for Time-Series Forecasting: Architectures, Symmetric Hyperparameter Evaluation, and a Nonlinear Benchmark

In this study, we developed and evaluated four conditional energy-based forecasting architectures: a classical Gaussian-Bernoulli CRBM, a hybrid quantum-classical QCRBM, a full-register QQRBM, and a lag-feature QFeatureQRBM with complete derivations of their conditional distributions, Contrastive-Divergence gradients, and hybrid training, bridging the energy-based formulation and the implementation-level quantum computation. Unlike prior comparisons, our evaluation enforces symmetric hyperparameter optimisation: classical and quantum-specific hyperparameters receive an equally thorough grid search across thirteen structured experiments. We test on two data classes, a Gaussian-process dataset (GP) generated with real financial data and the input-driven NARMA-10 nonlinear benchmark. Across both regimes we find no systematic evidence of a quantum advantage at the available sample size: no quantum architecture improves on the best classical baseline. The fully quantum QQRBM and QFeatureQRBM are significantly worse, whereas the hybrid QCRBM is statistically indistinguishable from the strongest classical CRBM on both datasets. A power analysis bounds this null result: at n = 12 only medium-to-large effects are detectable, so small advantages cannot be excluded. An iso-parameter (matched-budget) comparison reaches the same conclusion: the classical CRBM is lowest at three of the four budgets and no CRBM-vs-QCRBM difference is significant at any budget.

Gerhard Hellstern, Danyal Maheshwari, Martin Zaefferer +2

Centre of Finance DHBW Stuttgart Stuttgart, Germany · Zentrum für Digitale Innovationen DHBW Ravensburg Ravensburg, Germany · DATEV eG Nürnberg, Germany

Time Series ForecastingForecasting BenchmarksQuantum Machine LearningHybrid Quantum-Classical MLRestricted Boltzmann Machines
AskPDF
Jul 25, 2026

Exploration of the generative capabilities of Boltzmann machines applied to social systems under the majority rule

We study the generative capabilities of Boltzmann machines to recover systems governed by the majority rule under critical conditions. To this end, we train deep belief networks (DBNs) with different configurations, where the first layer can use Gaussian visible units with more than two states (i.e., non-binary units). We then allow the DBN to "dream" samples conditioned on visible units that we keep fixed, and we measure the deviation of this dreamed system from the real one. We also corroborate, using a discrete thermometer based on a convolutional network, that the reconstructions remain in a critical state. Across several training sessions with different architectures, we show that, despite the complexity of the problem, the DBN can recover samples that remain critical even under input noise, with a gradual degradation of physical observables relative to the original sample.

Mauricio A. Valle, Gonzalo A. Ruz

Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science (SODAS) Santiago, Chile · Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science (SODAS) Millennium Nucleus in Data Science for Plant Resilience (PhytoLearning) Santiago, Chile

Generative ModelingRestricted Boltzmann Machines
AskPDF
Jul 22, 2026

Machine-Learned Compact Subspace Generation for Quantum Selected Configuration Interaction within Density Matrix Embedding Framework

Sample-based Quantum Diagonalization (SQD), an extension of Quantum Selected Configuration Interaction (QSCI), has emerged as a promising hybrid quantum-classical paradigm for computing molecular ground state energies. By leveraging quantum sampling instead of variational optimization, QSCI avoids barren plateaus and enables direct reconstruction of correlated electronic wavefunctions. However, existing configuration recovery techniques primarily enforce symmetry constraints without guaranteeing optimal selection of the most physically relevant configurations, often leading to unnecessarily large subspaces and increased classical diagonalization costs. In this work, we introduce a machine-learned compact subspace generation protocol based on Restricted Boltzmann Machines (RBMs), termed QSCI-RBM, and integrate it within the Density Matrix Embedding Theory (DMET) framework. The RBM is trained on quantum-sampled configurations to learn the underlying probability distribution of dominant determinants, enabling the targeted generation of high-probability configurations. We apply this framework to the simulation of a protein-ligand complex involving the inhibitor Carmofur bound to the SARS-CoV-2 main protease (MproM^{\text{pro}}). Our results demonstrate that DMET-QSCI-RBM achieves energies within the chemical accuracy threshold by accessing only approximately 4% of the configuration subspace. In contrast, standard DMET-SQD simulations failed to reach chemical accuracy while accessing up to 20% of the subspace, even as the chemical potential itself nearly converged. These findings highlight that RBM-assisted configuration generation produces significantly more compact subspaces while preserving physical accuracy, thereby reducing classical computational overhead and enabling the scalable quantum embedding simulation of complex biological systems.

Ashish Kumar Patra, Anurag K. S. V., Ruchika Bhat +3

1Qclairvoyance Quantum Labs, Secunderabad, TG 500094, India. · 2The University of Arizona, Tucson, AZ 85721, USA. · 3Indian Institute of Technology Bombay, Mumbai, MH 400076, India.

Quantum ChemistryRestricted Boltzmann Machines
AskPDF
Jul 14, 2026

PolarBM: Complex-valued Boltzmann Machine for Modeling Audio Signals in Polar and Log-polar Coordinates

Although vast amounts of data, such as audio signal spectra, are naturally represented using complex numbers, conventional machine learning methods often simplify complex-domain problems by employing frameworks designed for real-valued variables. While this simplification offers computational benefits, it discards structural information regarding the inherent relationship between amplitude and phase. In this paper, we propose a novel Boltzmann machine (BM), named PolarBM, capable of naturally handling complex-valued variables in the polar coordinate (i.e., an amplitude-phase representation). PolarBM defines a probability density function for complex variables in which the phase explicitly depends on the amplitude, thereby capturing the physically important relationships of complex-valued signals. Furthermore, to process audio signals in accordance with human auditory perception, we propose LogPolarBM, which models amplitude on a logarithmic scale. This extension yields a flexible conditional probability density function, a power-weighted noncentral complex Gaussian (PW-NCCG) distribution, whose marginal amplitude distribution encompasses the Rice, Nakagami, and noncentral chi distributions as special cases. For practical applications, we also introduce the restricted variants of these proposed models: PolarRBM and LogPolarRBM. Experimental results demonstrate that by explicitly modeling the dependency between amplitude and phase, the proposed RBMs achieve superior modeling accuracy compared to conventional models, including deep neural networks. Although our experiments focus on audio signals, the utility of the proposed BMs is not limited to audio applications; their potential extends widely across various fields of science and engineering that involve complex-valued data, such as wireless communications and quantum mechanics.

Toru Nakashika, Kohei Yatabe

Graduate School of Informatics and Engineering, the University of Electro-Communications, Tokyo 182-8585, Japan · Department of Electrical Engineering and Computer Science, Tokyo University of Agriculture and Technology, Tokyo 184-8588, Japan

Unsupervised Representation LearningComplex-Valued Neural NetworksRestricted Boltzmann Machines
AskPDF
Jul 11, 2026

Learning from Noise: Effective-Rank Collapse and Out-of-Distribution Rejection in Restricted Boltzmann Machines

Restricted Boltzmann machines (RBMs) represent data by shaping an energy landscape over visible and hidden configurations, but their discriminative use is fragile under out-of-distribution (OOD) inputs: samples outside the training distribution can be absorbed into one of the learned class basins rather than rejected. Here, we analyze this failure mode through the spectrum of the induced visible--visible interaction J=WWTJ=WW^{T}, where WW is the visible--hidden weight matrix. Relative to a Marchenko--Pastur random-matrix reference, conventional training spreads spectral weight into many weak, bulk-compatible directions, increasing the effective rank of JJ. When auxiliary random binary images are assigned to a rejection label during training, the learned interaction undergoes effective-rank collapse: weak bulk-like modes are depleted, spectral weight concentrates into fewer dominant eigendirections, and the effective rank of JJ approaches that of the empirical data covariance matrix. The resulting RBM rejects structured OOD image datasets while preserving MNIST classification accuracy, showing that random auxiliary exposure can reshape both the interaction spectrum and the free-energy landscape of an energy-based classifier.

Oshada Rathnayake, Nikhil Shukla

University of Virginia, Charlottesville, VA, 22903, USA

Energy-Based ModelsClassificationOOD DetectionRestricted Boltzmann Machines
AskPDF
Jun 30, 2026

Explaining Machine Learning and Memorization with Statistical Mechanics

Artificial neural networks (NNs) and machine learning (ML) algorithms are poorly understood from a theoretical perspective, which makes it difficult to fully realize their potential and overcome their weaknesses. For instance, ML algorithms train NN weights by moving them along a low-dimensional subspace of their allowed values, but this implicitly low-dimensional learning structure is not properly exploited to improve training because its nature is not well understood. Moreover, trained NNs are easily confused by pervasive adversarial attacks whose theoretical underpinnings are still unclear. This thesis aims to improve our theoretical understanding of NNs and ML, with a particular focus on adversarial attacks and implicitly low-dimensional learning. For this purpose, we use mathematical tools from statistical mechanics to study different types of NNs and ways in which they can fit the data. In particular, we study two classes of models that fit the data with various degrees of learning and memorization: dense associative memory (DAM) and restricted Boltzmann machines (RBM). In the process, we investigate connections between different versions of these models that are useful to make analytical investigations more efficient.

Robin Theriault

Faculty of Science

Statistical Physics of LearningNeural Network MemorizationAdversarial RobustnessMemorization in Generative ModelsStatistical Learning TheoryAssociative Memory+1
AskPDF
Jun 29, 2026

Bridging the NISQ and Fault-Tolerant Regimes: Generative-ML-Assisted Quantum Selected CI for Molecular Simulations

Calculation of binding energies for protein-ligand molecular systems requires accurate treatment of the electronic structure, a quantum chemistry problem that scales exponentially on classical hardware, while current quantum hardware remains too noisy for the required circuit depths. This report presents a hybrid quantum-classical workflow performed on the Fujitsu FX700 ideal state-vector simulator using QARP that addresses two structural inefficiencies in quantum-sampling-based diagonalization workflows. First, we integrate the Linear Scaling CNOT UCCSD (LCNot-UCCSD) ansatz into the QSCI framework, replacing the O(N6)\mathcal{O}(N^6) CCSD parameter initialization of the competing LUCJ ansatz approach with O(N4)\mathcal{O}(N^4) MP2-amplitude initialization. Second, we introduce QSCI-RBM, a variant that replaces the configuration recovery of the SQD framework with a Restricted Boltzmann Machine (RBM) acting as a compact generative subspace expansion model. Both are evaluated on eight different molecules in STO-3G across 14 controlled artificial error levels with 100 independent runs each, validated on potential energy surface scans of the N2_2 molecule in cc-pVDZ, and embedded within DMET to treat the FDA-approved antiviral Amantadine (C10_{10}H17_{17}N, 11 DMET fragments) and the active region of the SARS-CoV-2 main protease complexed with its covalent inhibitor Carmofur (PDB: 7BUY, C15_{15}H28_{28}N4_4O5_5S, 10 fragments). To our knowledge, this is the first deployment of LCNot-UCCSD within QSCI on a quantum computing simulator, and the first DMET-QSCI(LCNot-UCCSD)-RBM application to an industry-relevant protein-ligand system. By utilizing a fraction of the classical computing resources required by the current state-of-the-art work by Cleveland Clinic, RIKEN, and IBM Quantum, this approach enables more efficient and economical drug discovery simulations for the industry.

Anurag K. S. V., Ashish Kumar Patra, Manas Mukherjee +4

Qclairvoyance Quantum Labs, Secunderabad, TG 500094, India · Centre for Quantum Technologies, National University of Singapore, SG 117543, Singapore · The University of Arizona, Tucson, AZ 85721, USA +1

Quantum ChemistryRestricted Boltzmann MachinesDrug DiscoveryComputational Quantum Chemistry
AskPDF
May 18, 2026

Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines

The great success of neural networks primarily arises from the presence of the large number of weight parameters combined with nonlinearities in the input-output relationship of single neurons. In this work, we study the relationship between the statistical properties of the weights and the nonlinearity of the hidden unit in Restricted Boltzmann Machines (RBMs) on the one side, and the distribution induced on binary visible units. We do this for four commonly used activation functions: Linear, Step, ReLU, and Exponential, and make qualitative predictions about the ability of these models to learn distributions with strong higher order interactions over the visible nodes. We show that in general, in an ensemble of RBMs with Gaussian weights, these distributions are rare and hard to learn, except when the hidden unit activation function is an Exponential.

Giovanni di Sarra, Yasser Roudi

Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, 7491 Trondheim, Norway · Department of Mathematics, King’s College London, London, Strand, WC2R 2LS, UK

Unsupervised Representation LearningNeural Network Activation FunctionsRestricted Boltzmann Machines
AskPDF
May 8, 2026

Three-in-One World Model: Energy-Based Consistency, Prediction, and Counterfactual Inference for Marketing Intervention

Marketing decisions reflect the interaction of latent consumer heterogeneity, time-varying internal states, and explicit interventions, a structure that current prediction- and language-oriented models do not capture in a unified manner. We propose a Three-in-One world-model architecture in which a Deep Boltzmann Machine (DBM) learns a frozen belief representation from demographics, time, and lagged actions and outcomes, with lightweight task-specific adapters attached on top. The same belief supports three tasks within a single framework: (i) energy-based consistency evaluation through the DBM's free energy, (ii) outcome prediction through adapters, and (iii) counterfactual inference by holding the belief fixed and varying only the action input given to the adapter. Using a controlled simulation in which the latent price sensitivity, promotion responsiveness, and base preference of each consumer are known, we show that the adapters match a strong MLP baseline on visit- and purchase-AUC while recovering heterogeneous treatment effects substantially better than S-, T-, X-, and DR-learner meta-learners and a Causal Forest baseline built on the same raw features, with the largest gap on a confounded price-promotion intervention. Complementing this, free-energy clamps systematically penalize counterfactual purchase trajectories that lack prior promotional exposure, and the penalty itself depends on the latent base preference in the expected direction. These results indicate that DBM beliefs disentangle latent traits in a form that survives counterfactual queries, providing an integrated world-model substrate for marketing intervention.

Junichiro Niimi

1Meijo University

Causal Effect EstimationWorld Model LearningConditional Average Treatment Effect EstimationEnergy-Based ModelsCounterfactual ReasoningRestricted Boltzmann Machines+2
AskPDF
May 7, 2026

Lecture Notes on Statistical Physics and Neural Networks

These lecture notes introduce some topics of classical statistical physics, particularly those that are relevant for neural networks and deep learning. Statistical physics is treated as a branch of probability theory or statistics, with the goal of making concepts such as phase transitions and the renormalization group accessible to readers without prior knowledge of physics. We introduce the Boltzmann-Gibbs distribution and the thermodynamic potentials on a finite configuration space, notably for Ising spins and spin-glass models on a lattice, and then define phase transitions as discontinuities that arise in the limit that the number of lattice points goes to infinity. We further introduce Hopfield networks and Boltzmann machines, which are governed by the same energy function as spin-glass models, and discuss the learning algorithm for restricted Boltzmann machines. In this algorithm hidden neurons are integrated out as in the renormalization group. Finally, modern deep learning is introduced, whose early developments were in part motivated by restricted Boltzmann machines in that they carry many layers of hidden neurons. A description of large language models is given.

Olaf Hohm

Institute for Physics, Humboldt University Berlin, Zum Großen Windkanal 2, D-12489 Berlin, Germany

Phase TransitionsRestricted Boltzmann Machines
AskPDF
Apr 1, 2026

Rapid mixing in positively weighted restricted Boltzmann machines

We show polylogarithmic mixing time bounds for the alternating-scan sampler for positively weighted restricted Boltzmann machines. This is done via analysing the same chain and the Glauber dynamics for ferromagnetic two-spin systems, where we obtain new mixing time bounds up to the critical thresholds.

Weiming Feng, Heng Guo, Minji Yang

School of Computing and Data Science, The University of Hong Kong, HK · School of Informatics, University of Edinburgh, Informatics Forum, UK

Restricted Boltzmann MachinesMarkov Chain Monte Carlo
AskPDF
Mar 3, 2026

Thermodynamic Regulation of Finite-Time Gibbs Training in Energy-Based Models: A Restricted Boltzmann Machine Study

Restricted Boltzmann Machines (RBMs) are typically trained using finite-length Gibbs chains under a fixed sampling temperature. This practice implicitly assumes that the stochastic regime remains valid as the energy landscape evolves during learning. We argue that this assumption can become structurally fragile under finite-time training dynamics. This fragility arises because, in nonconvex energy-based models, fixed-temperature finite-time training can generate admissible trajectories with effective-field amplification and conductance collapse. As a result, the Gibbs sampler may asymptotically freeze, the negative phase may localize, and, without sufficiently strong regularization, parameters may exhibit deterministic linear drift. To address this instability, we introduce an endogenous thermodynamic regulation framework in which temperature evolves as a dynamical state variable coupled to measurable sampling statistics. Under standard local Lipschitz conditions and a two-time-scale separation regime, we establish global parameter boundedness under strictly positive L2 regularization. We further prove local exponential stability of the thermodynamic subsystem and show that the regulated regime mitigates inverse-temperature blow-up and freezing-induced degeneracy within a forward-invariant neighborhood. Experiments on MNIST demonstrate that the proposed self-regulated RBM substantially improves normalization stability and effective sample size relative to fixed-temperature baselines, while preserving reconstruction performance. Overall, the results reinterpret RBM training as a controlled non-equilibrium dynamical process rather than a static equilibrium approximation.

Görkem Can Süleymanoğlu

Owner and Software Developer, Kuanka Publishing LLC, Turkey. · Ph.D. Student in Economics, Selçuk University, Institute of Social Sciences, Turkey.

Restricted Boltzmann MachinesEnergy-Based ModelsStatistical Physics of LearningMarkov Chain Monte Carlo
AskPDF
Aug 13, 2025

Comparison of D-Wave Quantum Annealing and Gibbs Monte Carlo for Sampling from a Probability Distribution of a Restricted Boltzmann Machine

A local-valley (LV) centered approach to assessing the quality of sampling from Restricted Boltzmann Machines (RBMs) was applied to the latest generation of the D-Wave quantum annealer. D-Wave and Gibbs samples from a classically trained RBM were obtained at conditions relevant to the contrastive-divergence-based RBM learning. The samples were compared for the number of the LVs to which they belonged and the energy of the corresponding local minima. No significant (desirable) increase in the number of the LVs has been achieved by decreasing the D-Wave annealing time. At any training epoch, the states sampled by the D-Wave belonged to a somewhat higher number of LVs than in the Gibbs sampling. However, many of those LVs found by the two techniques differed. For high-probability sampled states, the two techniques were (unfavorably) less complementary and more overlapping. Nevertheless, many potentially "important" local minima, i.e., those having intermediate, even if not high, probability values, were found by only one of the two sampling techniques while missed by the other. The two techniques overlapped less at later than earlier training epochs, which is precisely the stage of the training when modest improvements to the sampling quality could make meaningful differences for the RBM trainability. The results of this work may explain the failure of previous investigations to achieve substantial (or any) improvement when using D-Wave-based sampling. However, the results reveal some potential for improvement, e.g., using a combined classical-quantum approach.

Abdelmoula El-Yazizi, Yaroslav Koshka

Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA

Markov Chain Monte CarloQuantum AnnealingQuantum Machine LearningUnsupervised Representation LearningRestricted Boltzmann Machines
AskPDF