Textbooks on Various Subjects

I believe well-written textbooks (or even theses) are the fastest way to learn topics that have achieved critical mass. Inspired by a similarly titled post on LessWrong 2.0, I have my own evolving list.

For obvious reasons, I haven't read most books cover to cover. As I read more, I might start providing broader context towards - what can and should be read, what can be skipped, at what stage of the career to read it and so on, to make this list more meaningful. The years are only indicative and by all means prefer the latest edition/print (I've mentioned both first and latest prints).

General Math

• Mathematical Methods for Physics and Engineering by K. F. Riley, ‎M. P. Hobson, ‎S. J. Bence (2006)

Linear Algebra

• Introduction to Linear Algebra by Gilbert Strang (1993; 2016)
• Linear Algebra Done Right by Sheldon Axler (1995; 2015)

Optimization

• Numerical Optimization by Jorge Nocedal, Stephen J. Wright (2000; 2006)
• Convex Optimization by Stephen Boyd and Lieven Vandenberghe (2004)

Bayesian Inference

• Bayesian Data Analysis by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin (1995; 2013)
• Bayesian Reasoning and Machine Learning by David Barber (2012)

Monte Carlo Methods

• Handbook of Monte Carlo Methods by D.P. Kroese, T. Taimre, Z.I. Botev (2011)
• Handbook of Markov Chain Monte Carlo by various authors; edited by Steve Brooks, Andrew Gelman, Galin L. Jones and Xiao-Li Meng (2011)
• Monte Carlo theory, methods and examples by Art Owen (2013)

Information Theory

• Information Theory, Inference and Learning Algorithms by David J. C. MacKay (2003)

Stochastic Processes

• Applied Stochastic Differential Equations by Simo Särkkä and Arno Solin (2019)

Machine Learning / Statistics

• All of Statistics: A Concise Course in Statistical Inference by Larry Wasserman (2004)
• Pattern Recognition and Machine Learning by Christopher Bishop (2006; 2011)
• Machine Learning: a Probabilistic Perspective by Kevin Patrick Murphy (2012)

Gaussian Processes

• Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams (2006)

Deep Learning

• Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016)

Reinforcement Learning

• Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (1998; 2018)
• Reinforcement Learning: Theory and Algorithms by Alekh Agarwal, Nan Jiang and Sham M. Kakade (2019)

Learning Theory

• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2012; 2018)
• Understanding Machine Learning: From Theory to Algorithms ****by Shai Shalev-Shwartz and Shai Ben-David (2014)