Neural network models can now recognise images, understand text, translatelanguages, and play many human games at human or superhuman levels. Thesesystems are highly abstracted, but are inspired by biological brains and useonly biologically plausible computations. In the coming years, neural networksare likely to become less reliant on learning from massive labelled datasets,and more robust and generalisable in their task performance. From theirsuccesses and failures, we can learn about the computational requirements ofthe different tasks at which brains excel. Deep learning also provides thetools for testing cognitive theories. In order to test a theory, we need torealise the proposed information-processing system at scale, so as to be ableto assess its feasibility and emergent behaviours. Deep learning allows us toscale up from principles and circuit models to end-to-end trainable modelscapable of performing complex tasks. There are many levels at which cognitiveneuroscientists can use deep learning in their work, from inspiring theories toserving as full computational models. Ongoing advances in deep learning bringus closer to understanding how cognition and perception may be implemented inthe brain -- the grand challenge at the core of cognitive neuroscience.