About this resource
Nowadays, nearly anyone can implement a deep learning model in a just a few lines of code. What separates the novices from the experts, however, is the ability to understand (or at least predict!) how these models work in different circumstances.
Simon J.D. Prince’s free textbook, Understanding Deep Learning, provides a modern overview of deep learning (including newer topics like double descent and transformer models), and provides colab notebooks (!!!) to explore deep learning concepts and implementations. The book uses PyTorch as its framework of choice.
From the author, Simon J.D. Prince
The title of this book is “Understanding Deep Learning” to distinguish it from volumes that cover coding and other practical aspects. This text is primarily about the ideas that underlie deep learning. The first part of the book introduces deep learning models and discusses how to train them, measure their performance, and improve this performance. The next part considers architectures that are specialized to images, text, and graph data. These chapters require only introductory linear algebra, calculus, and probability and should be accessible to any second-year undergraduate in a quantitative discipline. Subsequent parts of the book tackle generative models and reinforcement learning. These chapters require more knowledge of probability and calculus and target more advanced students. The title is also partly a joke — no-one really understands deep learning at the time of writing. Modern deep networks learn piecewise linear functions with more regions than there are atoms in the universe and can be trained with fewer data examples than model parameters. It is neither obvious that we should be able to fit these functions reliably nor that they should generalize well to new data. The penultimate chapter addresses these and other aspects that are not yet fully understood. Regardless, deep learning will change the world for better or worse. The final chapter discusses AI ethics and concludes with an appeal for practitioners to consider the moral implications of their work.
Prerequisites
Learners are expected to have the following knowledge:
- Linear algebra: linear algebra is the language of machine learning
- Calculus: recommended to understand gradient descent
- Probability theory: needed for reinforcement learning
- PyTorch: recommended for following along with Colab notebooks
Estimated time to complete
TBD: Use the Improve this page functionality to add your own estimate!
Questions?
If you any lingering questions about this resource, please feel free to post to the Nexus Q&A on GitHub. We will improve materials on this website as additional questions come in.