Abstract
Most existing video summarisation methods are based on either supervised orunsupervised learning. In this paper, we propose a reinforcement learning-basedweakly supervised method that exploits easy-to-obtain, video-level categorylabels and encourages summaries to contain category-related information andmaintain category recognisability. Specifically, We formulate videosummarisation as a sequential decision-making process and train a summarisationnetwork with deep Q-learning (DQSN). A companion classification network is alsotrained to provide rewards for training the DQSN. With the classificationnetwork, we develop a global recognisability reward based on the classificationresult. Critically, a novel dense ranking-based reward is also proposed inorder to cope with the temporally delayed and sparse reward problems for longsequence reinforcement learning. Extensive experiments on two benchmarkdatasets show that the proposed approach achieves state-of-the-art performance.