Backprop Evolution

  • 2018-08-08 15:23:14
  • Maximilian Alber, Irwan Bello, Barret Zoph, Pieter-Jan Kindermans, Prajit Ramachandran, Quoc Le
  • 67


The back-propagation algorithm is the cornerstone of deep learning. Despiteits importance, few variations of the algorithm have been attempted. This workpresents an approach to discover new variations of the back-propagationequation. We use a domain specific lan- guage to describe update equations as alist of primitive functions. An evolution-based method is used to discover newpropagation rules that maximize the generalization per- formance after a fewepochs of training. We find several update equations that can train faster withshort training times than standard back-propagation, and perform similar asstandard back-propagation at convergence.


Introduction (beta)



Conclusion (beta)