Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles

  • 2023-06-01 18:59:58
  • Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, Chen Wei, Haoqi Fan, Po-Yao Huang, Vaibhav Aggarwal, Arkabandhu Chowdhury, Omid Poursaeed, Judy Hoffman, Jitendra Malik, Yanghao Li, Christoph Feichtenhofer
  • 0


Modern hierarchical vision transformers have added several vision-specificcomponents in the pursuit of supervised classification performance. While thesecomponents lead to effective accuracies and attractive FLOP counts, the addedcomplexity actually makes these transformers slower than their vanilla ViTcounterparts. In this paper, we argue that this additional bulk is unnecessary.By pretraining with a strong visual pretext task (MAE), we can strip out allthe bells-and-whistles from a state-of-the-art multi-stage vision transformerwithout losing accuracy. In the process, we create Hiera, an extremely simplehierarchical vision transformer that is more accurate than previous modelswhile being significantly faster both at inference and during training. Weevaluate Hiera on a variety of tasks for image and video recognition. Our codeand models are available at https://github.com/facebookresearch/hiera.


Quick Read (beta)

loading the full paper ...