Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Stochastic Optimal Control Matching
(Submitted on 4 Dec 2023 (v1), last revised 17 Apr 2024 (this version, v3))
Abstract: Stochastic optimal control, which has the goal of driving the behavior of noisy systems, is broadly applicable in science, engineering and artificial intelligence. Our work introduces Stochastic Optimal Control Matching (SOCM), a novel Iterative Diffusion Optimization (IDO) technique for stochastic optimal control that stems from the same philosophy as the conditional score matching loss for diffusion models. That is, the control is learned via a least squares problem by trying to fit a matching vector field. The training loss, which is closely connected to the cross-entropy loss, is optimized with respect to both the control function and a family of reparameterization matrices which appear in the matching vector field. The optimization with respect to the reparameterization matrices aims at minimizing the variance of the matching vector field. Experimentally, our algorithm achieves lower error than all the existing IDO techniques for stochastic optimal control for three out of four control problems, in some cases by an order of magnitude. The key idea underlying SOCM is the path-wise reparameterization trick, a novel technique that may be of independent interest. Code at this https URL
Submission history
From: Carles Domingo-Enrich [view email][v1] Mon, 4 Dec 2023 16:49:43 GMT (6048kb,D)
[v2] Thu, 14 Dec 2023 06:44:31 GMT (4889kb,D)
[v3] Wed, 17 Apr 2024 21:39:34 GMT (10053kb,D)
Link back to: arXiv, form interface, contact.