Textbooks on Various Subjects

Well-written textbooks (or even theses) are the fastest way to learn technical topics that have achieved critical mass. Inspired by a similarly titled post on LessWrong, I have my own evolving list.

For obvious reasons, I have not read most books cover to cover. I have, however, read a few chapters of each to be convinced that the rest of the book would be worth reading. Often, multiple books cater to overlapping topics, and provide complementary strengths to aid understanding. When multiple books are specified within each (sub-)section, it is safe to assume that as a "soft" recommendation order.

General Mathematics

• Mathematical Methods for Physics and Engineering by K. F. Riley, ‎M. P. Hobson, ‎S. J. Bence (2006)

Linear Algebra

• Linear Algebra Done Right by Sheldon Axler (1995; 2015)

• Introduction to Linear Algebra by Gilbert Strang (1993; 2016)

Numerical Methods

• Numerical Linear Algebra by Lloyd N. Trefethen and David Bau, III (1997)

• Fundamentals of Matrix Computations by David S. Watkins (2010)

• Matrix Computations by G.H. Golub and C.F. Van Loan (2013)

Optimization

• Convex Optimization by Stephen Boyd and Lieven Vandenberghe (2004)

• Numerical Optimization by Jorge Nocedal, Stephen J. Wright (2000; 2006)

Differential Equations

• Introduction to Partial Differential Equations by Peter J. Olver (2014; 2016)

• Partial Differential Equations: An Introduction by Walter A. Strauss (2008)

Machine Learning / Statistics

• Pattern Recognition and Machine Learning by Christopher Bishop (2006)

• Patterns, Predictions, and Actions: A Story about Machine Learning by Moritz Hardt and Benjamin Recht (2021)

Bayesian Inference

• Information Theory, Inference and Learning Algorithms by David J. C. MacKay (2003)

• Probabilistic Machine Learning by Kevin Murphy (book series 2012, 2021, 2022)

• Bayesian Reasoning and Machine Learning by David Barber (2012)

• Bayesian Data Analysis by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin (2020)

Gaussian Processes

• Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams (2006)

Deep Learning

• Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016)

Reinforcement Learning

• Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (1998; 2018)
• Reinforcement Learning: Theory and Algorithms by Alekh Agarwal, Nan Jiang and Sham M. Kakade (2019)

Learning Theory

• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2012; 2018)
• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David (2014)

Monte Carlo Methods

• Monte Carlo theory, methods and examples by Art Owen (2013)

• Handbook of Markov Chain Monte Carlo by various authors; edited by Steve Brooks, Andrew Gelman, Galin L. Jones and Xiao-Li Meng (2011)

• Handbook of Monte Carlo Methods by D.P. Kroese, T. Taimre, Z.I. Botev (2011)