This project implements reinforcement learning(RL) based on parameterized quantum circuits(PQCs) with quantum computing cloud Quafu. The work first searches a suitable architecture through four modules including variational PQC, data-encoding PQC, entanglement and measurement as shown in the follwing figure.
In this 4-qubit quantum circuit,
During evolutionary process, multi-objective is set to gain higher performance in RL task and reduce entanglement to achieve lower depth of compiled circuits.
Then, according to the effective architecture selected, one can train the model with Cirq simulator or Quafu backend. This file provides pre-trained model in './weights' of both ways and visualize them with animation in './visualization'. Noticeablely, the RL environment is chosen to CartPole-v1.
If you want to use the pre-trained model directly, the jupyter notebook in repo shows how to interact with Quafu and can output a gif if you have an access to display or you can refer to './validation/quantum_test.py' to infer pre-trained model and get corresponding records in a 'log.txt'.
If you want to train the model from scratch, you may need download this repo and set some paths to run the whole process, it's important to get your own token by registering on http://quafu.baqis.ac.cn/ .
Evolutionary Quantum Architecture Search for Parametrized Quantum Circuits, L. Ding and L. Spector, arXiv:2208.11167.
NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm, Z. Lu, et al, arXiv:1810.03522.
Parametrized Quantum Policies for Reinforcement Learnining, S. Jerbi, et al, arXiv:2103.05577.
https://github.com/ianwhale/nsga-net
https://tensorflow.google.cn/quantum/tutorials/quantum_reinforcement_learning