A Short Note on the Jensen-Shannon Divergence between Simple Mixture Distributions
share
This short note presents results about the symmetric Jensen-Shannon divergence between two discrete mixture distributions p_1 and p_2. Specifically, for i=1,2, p_i is the mixture of a common distribution q and a distribution p̃_i with mixture proportion λ_i. In general, p̃_1≠p̃_2 and λ_1≠λ_2. We provide experimental and theoretical insight to the behavior of the symmetric Jensen-Shannon divergence between p_1 and p_2 as the mixture proportions or the divergence between p̃_1 and p̃_2 change. We also provide insight into scenarios where the supports of the distributions p̃_1, p̃_2, and q do not coincide.
READ FULL TEXT