Abstract
Sports analytics has received significant attention from both academia andindustry in recent years. Despite the growing interest and efforts in thisfield, several issues remain unresolved, including (1) data unavailability, (2)lack of an effective trajectory-based framework, and (3) requirement forsufficient supervision labels. In this paper, we present TrajSV, atrajectory-based framework that addresses various issues in existing studies.TrajSV comprises three components: data preprocessing, Clip RepresentationNetwork (CRNet), and Video Representation Network (VRNet). The datapreprocessing module extracts player and ball trajectories from sportsbroadcast videos. CRNet utilizes a trajectory-enhanced Transformer module tolearn clip representations based on these trajectories. Additionally, VRNetlearns video representations by aggregating clip representations and visualfeatures with an encoder-decoder architecture. Finally, a triple contrastiveloss is introduced to optimize both video and clip representations in anunsupervised manner. The experiments are conducted on three broadcast videodatasets to verify the effectiveness of TrajSV for three types of sports (i.e.,soccer, basketball, and volleyball) with three downstream applications (i.e.,sports video retrieval, action spotting, and video captioning). The resultsdemonstrate that TrajSV achieves state-of-the-art performance in sports videoretrieval, showcasing a nearly 70% improvement. It outperforms baselines inaction spotting, achieving state-of-the-art results in 9 out of 17 actioncategories, and demonstrates a nearly 20% improvement in video captioning.Additionally, we introduce a deployed system along with the three applicationsbased on TrajSV.