Discriminative Mutual Information Estimators for Channel Capacity Learning

by   Nunzio A. Letizia, et al.

Channel capacity plays a crucial role in the development of modern communication systems as it represents the maximum rate at which information can be reliably transmitted over a communication channel. Nevertheless, for the majority of channels, finding a closed-form capacity expression remains an open challenge. This is because it requires to carry out two formidable tasks a) the computation of the mutual information between the channel input and output, and b) its maximization with respect to the signal distribution at the channel input. In this paper, we address both tasks. Inspired by implicit generative models, we propose a novel cooperative framework to automatically learn the channel capacity, for any type of memory-less channel. In particular, we firstly develop a new methodology to estimate the mutual information directly from a discriminator typically deployed to train adversarial networks, referred to as discriminative mutual information estimator (DIME). Secondly, we include the discriminator in a cooperative channel capacity learning framework, referred to as CORTICAL, where a discriminator learns to distinguish between dependent and independent channel input-output samples while a generator learns to produce the optimal channel input distribution for which the discriminator exhibits the best performance. Lastly, we prove that a particular choice of the cooperative value function solves the channel capacity estimation problem. Simulation results demonstrate that the proposed method offers high accuracy.


Cooperative Channel Capacity Learning

In this paper, the problem of determining the capacity of a communicatio...

Discriminative Mutual Information Estimation for the Design of Channel Capacity Driven Autoencoders

The development of optimal and efficient machine learning-based communic...

Capacity-Approaching Autoencoders for Communications

The autoencoder concept has fostered the reinterpretation and the design...

Neural Capacity Estimators: How Reliable Are They?

Recently, several methods have been proposed for estimating the mutual i...

MIND: Maximum Mutual Information Based Neural Decoder

We are assisting at a growing interest in the development of learning ar...

The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions

The mutual information is analyzed as a function of the input distributi...

Neural Mutual Information Estimation for Channel Coding: State-of-the-Art Estimators, Analysis, and Performance Comparison

Deep learning based physical layer design, i.e., using dense neural netw...

Please sign up or login with your details

Forgot password? Click here to reset