Abstract
In recent years, the transformer architecture has become the de factostandard for machine learning algorithms applied to natural language processingand computer vision. Despite notable evidence of successful deployment of thisarchitecture in the context of robot learning, we claim that vanillatransformers do not fully exploit the structure of the robot learning problem.Therefore, we propose Body Transformer (BoT), an architecture that leveragesthe robot embodiment by providing an inductive bias that guides the learningprocess. We represent the robot body as a graph of sensors and actuators, andrely on masked attention to pool information throughout the architecture. Theresulting architecture outperforms the vanilla transformer, as well as theclassical multilayer perceptron, in terms of task completion, scalingproperties, and computational efficiency when representing either imitation orreinforcement learning policies. Additional material including the open-sourcecode is available at https://sferrazza.cc/bot_site.