Faster Sampling from Log-Concave Distributions over Polytopes via a Soft-Threshold Dikin Walk
share
We consider the problem of sampling from a d-dimensional log-concave distribution π(θ) ∝ e^-f(θ) constrained to a polytope K defined by m inequalities. Our main result is a "soft-threshold” variant of the Dikin walk Markov chain that requires at most O((md + d L^2 R^2) × md^ω-1) log(w/δ)) arithmetic operations to sample from π within error δ>0 in the total variation distance from a w-warm start, where L is the Lipschitz-constant of f, K is contained in a ball of radius R and contains a ball of smaller radius r, and ω is the matrix-multiplication constant. When a warm start is not available, it implies an improvement of Õ(d^3.5-ω) arithmetic operations on the previous best bound for sampling from π within total variation error δ, which was obtained with the hit-and-run algorithm, in the setting where K is a polytope given by m=O(d) inequalities and LR = O(√(d)). When a warm start is available, our algorithm improves by a factor of d^2 arithmetic operations on the best previous bound in this setting, which was obtained for a different version of the Dikin walk algorithm. Plugging our Dikin walk Markov chain into the post-processing algorithm of Mangoubi and Vishnoi (2021), we achieve further improvements in the dependence of the running time for the problem of generating samples from π with infinity distance bounds in the special case when K is a polytope.
READ FULL TEXT