Abstract
Tremendous amounts of expensive annotated data are a vital ingredient forstate-of-the-art 3d hand pose estimation. Therefore, synthetic data has beenpopularized as annotations are automatically available. However, models trainedonly with synthetic samples do not generalize to real data, mainly due to thegap between the distribution of synthetic and real data. In this paper, wepropose a novel method that seeks to predict the 3d position of the hand usingboth synthetic and partially-labeled real data. Accordingly, we form a sharedlatent space between three modalities: synthetic depth image, real depth image,and pose. We demonstrate that by carefully learning the shared latent space, wecan find a regression model that is able to generalize to real data. As such,we show that our method produces accurate predictions in both semi-supervisedand unsupervised settings. Additionally, the proposed model is capable ofgenerating novel, meaningful, and consistent samples from all of the threedomains. We evaluate our method qualitatively and quantitively on two highlycompetitive benchmarks (i.e., NYU and ICVL) and demonstrate its superiorityover the state-of-the-art methods. The source code will be made available athttps://github.com/masabdi/LSPS.