Abstract
We propose a simple yet effective approach for few-shot action recognition,emphasizing the disentanglement of motion and appearance representations. Byharnessing recent progress in tracking, specifically point trajectories andself-supervised representation learning, we build trajectory-aligned tokens(TATs) that capture motion and appearance information. This approachsignificantly reduces the data requirements while retaining essentialinformation. To process these representations, we use a Masked Space-timeTransformer that effectively learns to aggregate information to facilitatefew-shot action recognition. We demonstrate state-of-the-art results onfew-shot action recognition across multiple datasets. Our project page isavailable at https://www.cs.umd.edu/~pulkit/tats