Control of Continuous Quantum Systems with Many Degrees of Freedom based on Convergent Reinforcement Learning

by   Zhikang Wang, et al.

With the development of experimental quantum technology, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems. However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and numerical algorithms which search for proper control protocols are adopted, and, deep learning, especially deep reinforcement learning (RL), is a promising generic candidate solution for the control problems. Although there have been a few successful applications of deep RL to quantum control problems, most of the existing RL algorithms suffer from instabilities and unsatisfactory reproducibility, and require a large amount of fine-tuning and a large computational budget, both of which limit their applicability. To resolve the issue of instabilities, in this dissertation, we investigate the non-convergence issue of Q-learning. Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm. We prove the convergence of C-DQN and apply it to the Atari 2600 benchmark. We show that when DQN fail, C-DQN still learns successfully. Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum quartic oscillator and a trapped quantum rigid body. We establish the physical models and analyse their properties, and we show that although both C-DQN and DQN can learn to cool the systems, C-DQN tends to behave more stably, and when DQN suffers from instabilities, C-DQN can achieve a better performance. As the performance of DQN can have a large variance and lack consistency, C-DQN can be a better choice for researches on complicated control problems.


page 1

page 2

page 3

page 4


Deep Reinforcement Learning Control of Quantum Cartpoles

We generalize a standard benchmark of reinforcement learning, the classi...

A Convergent and Efficient Deep Q Network Algorithm

Despite the empirical success of the deep Q network (DQN) reinforcement ...

Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks

Variational quantum eigensolvers have recently received increased attent...

Reinforcement learning architecture for automated quantum-adiabatic-algorithm design

Quantum algorithm design lies in the hallmark of applications of quantum...

Reinforcement Learning for Many-Body Ground State Preparation based on Counter-Diabatic Driving

The Quantum Approximate Optimization Ansatz (QAOA) is a prominent exampl...

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

Deep Reinforcement Learning (RL) has considerably advanced over the past...

Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning

Efficient quantum compiling tactics greatly enhance the capability of qu...

Please sign up or login with your details

Forgot password? Click here to reset