Abstract
Event-based cameras have shown great promise in a variety of situations whereframe based cameras suffer, such as high speed motions and high dynamic rangescenes. However, developing algorithms for event measurements requires a newclass of hand crafted algorithms. Deep learning has shown great success inproviding model free solutions to many problems in the vision community, butexisting networks have been developed with frame based images in mind, andthere does not exist the wealth of labeled data for events as there does forimages for supervised training. To these points, we present EV-FlowNet, a novelself-supervised deep learning pipeline for optical flow estimation for eventbased cameras. In particular, we introduce an image based representation of agiven event stream, which is fed into a self-supervised neural network as thesole input. The corresponding grayscale images captured from the same camera atthe same time as the events are then used as a supervisory signal to provide aloss function at training time, given the estimated flow from the network. Weshow that the resulting network is able to accurately predict optical flow fromevents only in a variety of different scenes, with performance competitive toimage based networks. This method not only allows for accurate estimation ofdense optical flow, but also provides a framework for the transfer of otherself-supervised methods to the event-based domain.