Encoding of musical structures in hidden units of restricted Boltzmann machines
Organizations: Department of Applied Physics and Physico-Informatics, Faculty of Science and Technology, Keio University, Yokohama 223-8522, Japan
Abstract
Restricted Boltzmann machines (RBMs) are energy-based models originating from statistical physics. In this study, we investigate how musical structures are represented in the hidden layer of a Bernoulli-Bernoulli RBM trained on binary piano-roll representations of J.~S.~Bach's music. By examining the visible patterns induced by individual hidden units, we find that the hidden layer primarily captures local temporal and pitch-statistical features rather than explicit musical concepts such as melodies, chords, or keys. Free-energy comparisons show that the trained RBM assigns higher probabilities to the examined musical inputs than to digit or noise patterns. Analysis using t-SNE, cosine similarity, and PCA reveals that hidden representations are sensitive to absolute pitch and input overlap, with transposed versions of the same piece not necessarily mapped close together. Generated samples exhibit local pitch organization but limited long-range coherence. These findings illustrate how a simple statistical-physics model captures certain regularities in structured musical data while highlighting its limitations in representing higher-level musical relationships.
Figures & tables
| Image size | |
|---|---|
| Number of visible units | 13,824 |
| Number of hidden units | 2048 |
| Training images | 21,444 |
| Training method | CD-1 |
| Training epochs | 1,000 |
| Configured batch size | 500 |
| Input | Active pixels | Joint energy | Visible free energy |
|---|---|---|---|
| Piano roll (trained) | 564 | ||
| Piano roll (untrained) | 360 | ||
| MNIST digit 0 | 2581 | ||
| MNIST digit 5 | 1436 | ||
| MNIST digit 8 | 1956 | ||
| Noise | 7014 |
| Semitone | Whole tone | |||
|---|---|---|---|---|
| Pair | Input | Hidden | Input | Hidden |
| A–B | ||||
| A’–B’ | ||||
| A–A’ | ||||
| B–B’ | ||||
| A–B’ | ||||
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Representation Learning
Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.
See all 855 papers