Learn to See by Events: RGB Frame Synthesis from Event Cameras

  • 2018-12-05 15:22:47
  • Stefano Pini, Guido Borghi, Roberto Vezzani, Rita Cucchiara
  • 3


Event cameras are biologically-inspired sensors that gather the temporalevolution of the scene, capturing only pixel-wise brightness variations.Despite having multiple advantages with respect to traditional cameras, theiruse is still limited due to the difficult intelligibility and restrictedusability through traditional vision algorithms. To this aim, we present aframework which exploits the output of event cameras to synthesize RGB frames.In particular, the frame generation relies on an initial or a periodic set ofcolor key-frames and a sequence of intermediate event frames, i.e. gray-levelimages that integrate the brightness changes captured by the event cameraduring a short temporal slot. An adversarial architecture combined with arecurrent module is employed for the frame synthesis. Both traditional andevent-based datasets are adopted to assess the capabilities of the proposedarchitecture: pixel-wise and semantic metrics confirm the quality of thesynthesized images.


Introduction (beta)



Conclusion (beta)