This paper is an attempt to explain all the matrix calculus you need in orderto understand the training of deep neural networks. We assume no math knowledgebeyond what you learned in calculus 1, and provide links to help you refreshthe necessary math where needed. Note that you do not need to understand thismaterial before you start learning to train and use deep learning in practice;rather, this material is for those who are already familiar with the basics ofneural networks, and wish to deepen their understanding of the underlying math.Don't worry if you get stuck at some point along the way---just go back andreread the previous section, and try writing down and working through someexamples. And if you're still stuck, we're happy to answer your questions inthe Theory category at forums.fast.ai. Note: There is a reference section atthe end of the paper summarizing all the key matrix calculus rules andterminology discussed here.