Multi-Agent Learning in Network Zero-Sum Games is a Hamiltonian System

by   James P. Bailey, et al.

Zero-sum games are natural, if informal, analogues of closed physical systems where no energy/utility can enter or exit. This analogy can be extended even further if we consider zero-sum network (polymatrix) games where multiple agents interact in a closed economy. Typically, (network) zero-sum games are studied from the perspective of Nash equilibria. Nevertheless, this comes in contrast with the way we typically think about closed physical systems, e.g., Earth-moon systems which move perpetually along recurrent trajectories of constant energy. We establish a formal and robust connection between multi-agent systems and Hamiltonian dynamics -- the same dynamics that describe conservative systems in physics. Specifically, we show that no matter the size, or network structure of such closed economies, even if agents use different online learning dynamics from the standard class of Follow-the-Regularized-Leader, they yield Hamiltonian dynamics. This approach generalizes the known connection to Hamiltonians for the special case of replicator dynamics in two agent zero-sum games developed by Hofbauer. Moreover, our results extend beyond zero-sum settings and provide a type of a Rosetta stone (see e.g. Table 1) that helps to translate results and techniques between online optimization, convex analysis, games theory, and physics.


page 1

page 2

page 3

page 4


Fast Convergence of Optimistic Gradient Ascent in Network Zero-Sum Extensive Form Games

The study of learning in games has thus far focused primarily on normal ...

Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincaré Recurrence

We present a novel control-theoretic understanding of online optimizatio...

Cycles in adversarial regularized learning

Regularized learning is a fundamental technique in online optimization, ...

The Dynamics of Q-learning in Population Games: a Physics-Inspired Continuity Equation Model

Although learning has found wide application in multi-agent systems, its...

Online Learning in Periodic Zero-Sum Games

A seminal result in game theory is von Neumann's minmax theorem, which s...

Fast and Furious Learning in Zero-Sum Games: Vanishing Regret with Non-Vanishing Step Sizes

We show for the first time, to our knowledge, that it is possible to rec...

Self-similarity in the Kepler-Heisenberg problem

The Kepler-Heisenberg problem is that of determining the motion of a pla...

Please sign up or login with your details

Forgot password? Click here to reset