Attention to Entropic Communication

by   Torsten Enßlin, et al.

The concept of attention, numerical weights that emphasize the importance of particular data, has proven to be very relevant in artificial intelligence. Relative entropy (RE, aka Kullback-Leibler divergence) plays a central role in communication theory. Here we combine these concepts, attention and RE. RE guides optimal encoding of messages in bandwidth-limited communication as well as optimal message decoding via the maximum entropy principle (MEP). In the coding scenario, RE can be derived from four requirements, namely being analytical, local, proper, and calibrated. Weighted RE, used for attention steering in communications, turns out to be improper. To see how proper attention communication can emerge, we analyze a scenario of a message sender who wants to ensure that the receiver of the message can perform well-informed actions. If the receiver decodes the message using the MEP, the sender only needs to know the receiver's utility function to inform optimally, but not the receiver's initial knowledge state. In case only the curvature of the utility function maxima are known, it becomes desirable to accurately communicate an attention function, in this case a by this curvature weighted and re-normalized probability function. Entropic attention communication is here proposed as the desired generalization of entropic communication that permits weighting while being proper, thereby aiding the design of optimal communication protocols in technical applications and helping to understand human communication. For example, our analysis shows how to derive the level of cooperation expected under misaligned interests of otherwise honest communication partners.


page 7

page 10


The Capacity of Anonymous Communications

We consider the communication scenario where K transmitters are each con...

Algorithms for Persuasion with Limited Communication

The Bayesian persuasion paradigm of strategic communication models inter...

On Joint Detection and Decoding in Short-Packet Communications

We consider a communication problem in which the receiver must first det...

Achieving Covert Wireless Communications Using a Full-Duplex Receiver

Covert communications hide the transmission of a message from a watchful...

Low Entropy Communication in Multi-Agent Reinforcement Learning

Communication in multi-agent reinforcement learning has been drawing att...

Covert Communication With Mismatched Decoders

This paper considers the problem of covert communication with mismatched...

A Switch to the Concern of User: Importance Coefficient in Utility Distribution and Message Importance Measure

This paper mainly focuses on the utilization frequency in receiving end ...

Please sign up or login with your details

Forgot password? Click here to reset