A short note on an inequality between KL and TV
share
The goal of this short note is to discuss the relation between Kullback–Leibler divergence and total variation distance, starting with the celebrated Pinsker's inequality relating the two, before switching to a simple, yet (arguably) more useful inequality, apparently not as well known, due to Bretagnolle and Huber. We also discuss applications of this bound for minimax testing lower bounds.
READ FULL TEXT