This paper considers cooperative Multi-Agent Reinforcement Learning, focusingon emergent communication in settings where multiple pairs of independentlearners interact at varying frequencies. In this context, multiple distinctand incompatible languages can emerge. When an agent encounters a speaker of analternative language, there is a requirement for a period of adaptation beforethey can efficiently converse. This adaptation results in the emergence of anew language and the forgetting of the previous language. In principle, this isan example of the Catastrophic Forgetting problem which can be mitigated byenabling the agents to learn and maintain multiple languages. We takeinspiration from the Continual Learning literature and equip our agents withmulti-headed neural networks which enable our agents to be multi-lingual. Ourmethod is empirically validated within a referential MNIST based communicationgame and is shown to be able to maintain multiple languages where existingapproaches cannot.