Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?
(Submitted on 27 Jul 2023 (v1), revised 23 Oct 2023 (this version, v2), latest version 23 Apr 2024 (v5))
Abstract: We prove that black-box variational inference (BBVI) with control variates, particularly the sticking-the-landing (STL) estimator, converges at a geometric (traditionally called "linear") rate under perfect variational family specification. In particular, we prove a quadratic bound on the gradient variance of the STL estimator, one which encompasses misspecified variational families. Combined with previous works on the quadratic variance condition, this directly implies convergence of BBVI with the use of projected stochastic gradient descent. We also improve existing analysis on the regular closed-form entropy gradient estimators, which enables comparison against the STL estimator and provides explicit non-asymptotic complexity guarantees for both.
Submission history
From: Kyurae Kim [view email][v1] Thu, 27 Jul 2023 06:32:43 GMT (263kb)
[v2] Mon, 23 Oct 2023 19:19:11 GMT (298kb)
[v3] Wed, 21 Feb 2024 23:03:37 GMT (13054kb)
[v4] Sat, 9 Mar 2024 01:10:21 GMT (327kb)
[v5] Tue, 23 Apr 2024 01:58:11 GMT (313kb)
Link back to: arXiv, form interface, contact.