Abstract
Vision transformers have revolutionized computer vision, but theircomputational demands present challenges for training and deployment. Thispaper introduces LOTUS (LOttery Transformers with Ultra Sparsity), a novelmethod that leverages data lottery ticket selection and sparsity pruning toaccelerate vision transformer training while maintaining accuracy. Our approachfocuses on identifying and utilizing the most informative data subsets andeliminating redundant model parameters to optimize the training process.Through extensive experiments, we demonstrate the effectiveness of LOTUS inachieving rapid convergence and high accuracy with significantly reducedcomputational requirements. This work highlights the potential of combiningdata selection and sparsity techniques for efficient vision transformertraining, opening doors for further research and development in this area.