Abstract
Variational Quantum Algorithms have emerged as promising tools for solvingoptimization problems on quantum computers. These algorithms leverage aparametric quantum circuit called ansatz, where its parameters are adjusted bya classical optimizer with the goal of optimizing a certain cost function.However, a significant challenge lies in designing effective circuits foraddressing specific problems. In this study, we leverage the powerful andflexible Reinforcement Learning paradigm to train an agent capable ofautonomously generating quantum circuits that can be used as ansatzes invariational algorithms to solve optimization problems. The agent is trained ondiverse problem instances, including Maximum Cut, Maximum Clique and MinimumVertex Cover, built from different graph topologies and sizes. Our analysis ofthe circuits generated by the agent and the corresponding solutions shows thatthe proposed method is able to generate effective ansatzes. While our goal isnot to propose any new specific ansatz, we observe how the agent has discovereda novel family of ansatzes effective for Maximum Cut problems, which we call$R_{yz}$-connected. We study the characteristics of one of these ansatzes bycomparing it against state-of-the-art quantum algorithms across instances ofvarying graph topologies, sizes, and problem types. Our results indicate thatthe $R_{yz}$-connected circuit achieves high approximation ratios for MaximumCut problems, further validating our proposed agent. In conclusion, our studyhighlights the potential of Reinforcement Learning techniques in assistingresearchers to design effective quantum circuits which could have applicationsin a wide number of tasks.