Abstract
We present an approach that can reconstruct hands in 3D from monocular input.Our approach for Hand Mesh Recovery, HaMeR, follows a fully transformer-basedarchitecture and can analyze hands with significantly increased accuracy androbustness compared to previous work. The key to HaMeR's success lies inscaling up both the data used for training and the capacity of the deep networkfor hand reconstruction. For training data, we combine multiple datasets thatcontain 2D or 3D hand annotations. For the deep model, we use a large scaleVision Transformer architecture. Our final model consistently outperforms theprevious baselines on popular 3D hand pose benchmarks. To further evaluate theeffect of our design in non-controlled settings, we annotate existingin-the-wild datasets with 2D hand keypoint annotations. On this newly collecteddataset of annotations, HInt, we demonstrate significant improvements overexisting baselines. We make our code, data and models available on the projectwebsite: https://geopavlakos.github.io/hamer/.