research
∙
02/10/2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD
Cyclic and randomized stepsizes are widely used in the deep learning pra...
share
research
∙
11/29/2022
Penalized Langevin and Hamiltonian Monte Carlo Algorithms for Constrained Sampling
We consider the constrained sampling problem where the goal is to sample...
research
∙
05/13/2022
Heavy-Tail Phenomenon in Decentralized SGD
Recent theoretical studies have shown that heavy-tails can emerge in sto...
research
∙
07/01/2020
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Ham...
research
∙
05/25/2020
Fractional moment-preserving initialization schemes for training fully-connected neural networks
A common approach to initialization in deep neural networks is to sample...
research
∙
04/06/2020