Recurrent neural networks (RNNs) are a class of neural networks that haveemerged from the paradigm of artificial intelligence and has enabled lots ofinteresting advances in the field of natural language processing.Interestingly, these architectures were shown to be powerful ansatze toapproximate the ground state of quantum systems. Here, we build over theresults of [Phys. Rev. Research 2, 023358 (2020)] and construct a more powerfulRNN wave function ansatz in two dimensions. We use symmetry and annealing toobtain accurate estimates of ground state energies of the two-dimensional (2D)Heisenberg model, on the square lattice and on the triangular lattice. We showthat our method is superior to Density Matrix Renormalisation Group (DMRG) forsystem sizes larger than or equal to $14 \times 14$ on the triangular lattice.