Current browse context:
cond-mat.dis-nn
Change to browse by:
References & Citations
Condensed Matter > Disordered Systems and Neural Networks
Title: Exact training of Restricted Boltzmann machines on intrinsically low dimensional data
(Submitted on 19 Mar 2021 (v1), last revised 17 Nov 2021 (this version, v2))
Abstract: The restricted Boltzmann machine is a basic machine learning tool able, in principle, to model the distribution of some arbitrary dataset. Its standard training procedure appears however delicate and obscure in many respects. We bring some new insights to it by considering the situation where the data have low intrinsic dimension, offering the possibility of an exact treatment and revealing a fundamental failure of the standard training procedure. The reasons for this failure \textemdash~like the occurrence of first-order phase transitions during training~\textemdash \ are clarified thanks to a Coulomb interactions reformulation of the model. In addition a convex relaxation of the original optimization problem is formulated thereby resulting in a unique solution, obtained in precise numerical form on $d=1,2$ study cases, while a constrained linear regression solution can be conjectured on the basis of an information theory argument.
Submission history
From: Cyril Furtlehner [view email][v1] Fri, 19 Mar 2021 11:45:55 GMT (3268kb,D)
[v2] Wed, 17 Nov 2021 10:01:37 GMT (3402kb,D)
Link back to: arXiv, form interface, contact.