Thompson Sampling and Approximate Inference

08/14/2019
by   My Phan, et al.
0

We study the effects of approximate inference on the performance of Thompson sampling in the k-armed bandit problems. Thompson sampling is a successful algorithm for online decision-making but requires posterior inference, which often must be approximated in practice. We show that even small constant inference error (in α-divergence) can lead to poor performance (linear regret) due to under-exploration (for α<1) or over-exploration (for α>0) by the approximation. While for α > 0 this is unavoidable, for α≤ 0 the regret can be improved by adding a small amount of forced exploration even when the inference error is a large constant.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset