Abstract
Emergent multi-agent communication protocols are very different from naturallanguage and not easily interpretable by humans. We find that agents that wereinitially pretrained to produce natural language can also experiencedetrimental language drift: when a non-linguistic reward is used in agoal-based task, e.g. some scalar success metric, the communication protocolmay easily and radically diverge from natural language. We recast translationas a multi-agent communication game and examine auxiliary training constraintsfor their effectiveness in mitigating language drift. We show that acombination of syntactic (language model likelihood) and semantic (visualgrounding) constraints gives the best communication performance, allowingpre-trained agents to retain English syntax while learning to accurately conveythe intended meaning.