A Generalist Agent

  • 2022-05-12 17:03:26
  • Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas
  • 54

Abstract

Inspired by progress in large-scale language modeling, we apply a similarapproach towards building a single generalist agent beyond the realm of textoutputs. The agent, which we refer to as Gato, works as a multi-modal,multi-task, multi-embodiment generalist policy. The same network with the sameweights can play Atari, caption images, chat, stack blocks with a real robotarm and much more, deciding based on its context whether to output text, jointtorques, button presses, or other tokens. In this report we describe the modeland the data, and document the current capabilities of Gato.

 

Quick Read (beta)

loading the full paper ...