Reinforcement Learning Evaluation and Solution for the Feedback Capacity of the Ising Channel with Large Alphabet

by   Ziv Aharoni, et al.

We propose a new method to compute the feedback capacity of unifilar finite state channels (FSCs) with memory using reinforcement learning (RL). The feedback capacity was previously estimated using its formulation as a Markov decision process (MDP) with dynamic programming (DP) algorithms. However, their computational complexity grows exponentially with the channel alphabet size. Therefore, we use RL, and specifically its ability to parameterize value functions and policies with neural networks, to evaluate numerically the feedback capacity of channels with a large alphabet size. The outcome of the RL algorithm is a numerical lower bound on the feedback capacity, which is used to reveal the structure of the optimal solution. The structure is modeled by a graph-based auxiliary random variable that is utilized to derive an analytic upper bound on the feedback capacity with the duality bound. The capacity computation is concluded by verifying the tightness of the upper bound by testing whether it is BCJR invariant. We demonstrate this method on the Ising channel with an arbitrary alphabet size. For an alphabet size smaller than or equal to 8, we derive the analytic solution of the capacity. Next, the structure of the numerical solution is used to deduce a simple coding scheme that achieves the feedback capacity and serves as a lower bound for larger alphabets. For an alphabet size greater than 8, we present an upper bound on the feedback capacity. For an asymptotically large alphabet size, we present an asymptotic optimal coding scheme.


page 1

page 2

page 3

page 4


Computing the Feedback Capacity of Finite State Channels using Reinforcement Learning

In this paper, we propose a novel method to compute the feedback capacit...

Capacity of Finite-State Channels with Delayed Feedback

In this paper, we investigate the capacity of finite-state channels (FSC...

Graph-Based Encoders and their Performance for Finite-State Channels with Feedback

The capacity of unifilar finite-state channels in the presence of feedba...

Feedback Capacity and Coding for the (0,k)-RLL Input-Constrained BEC

The input-constrained binary erasure channel (BEC) with strictly causal ...

Finite-State Channels with Feedback and State Known at the Encoder

We consider finite state channels (FSCs) with feedback and state informa...

On The Reliability Function of Discrete Memoryless Multiple-Access Channel with Feedback

We derive a lower and upper bound on the reliability function of discret...

Adaptive Ensemble Q-learning: Minimizing Estimation Bias via Error Feedback

The ensemble method is a promising way to mitigate the overestimation is...

Please sign up or login with your details

Forgot password? Click here to reset