A visual introduction to information theory

06/16/2022

∙

by   Henry Pinkard, et al.

∙

0

∙

share

Though originally developed for communications engineering, information theory contains mathematical tools with numerous applications in science and engineering. These tools can be used to characterize the fundamental limits of data compression and transmission in the presence of noise. Here, we present a practical guide to key concepts in information theory, focusing on intuitions and providing visual explanations wherever possible. Our presentation assumes only a familiarity with basic probability theory.