Activity Detection with Latent Sub-event Hierarchy Learning

  • 2018-03-16 17:09:54
  • AJ Piergiovanni, Michael S. Ryoo
  • 12

Abstract

In this paper, we introduce a new convolutional layer named the TemporalGaussian Mixture (TGM) layer and present how it can be used to efficientlycapture temporal structure in continuous activity videos. Our layer is designedto allow the model to learn a latent hierarchy of sub-event intervals. Ourapproach is fully differentiable while relying on a significantly less numberof parameters, enabling its end-to-end training with standard backpropagation.We present our convolutional video models with multiple TGM layers for activitydetection. Our experiments on multiple datasets including Charades andMultiTHUMOS confirm the benefit of our TGM layers, illustrating that itoutperforms other models and temporal convolutions.

 

Quick Read (beta)

loading the full paper ...