Math foundations

Topics, in order

1. Mark "Vectors, span and basis" as done

   1.1 Vectors, span and basis

   • Watch

     Essence of Linear Algebra, Ch 1–2: Vectors; Linear combinations, span and basis vectors

     3Blue1Brown

   • Read

     Mathematics for Machine Learning, Ch 2 (vector spaces, linear independence, basis)

     Deisenroth, Faisal, Ong · free

2. Mark "Matrices as transformations; matrix multiplication" as done

   1.2 Matrices as transformations; matrix multiplication

   • Watch

     Essence of Linear Algebra, Ch 3–4: Linear transformations and matrices; Matrix multiplication as composition

     3Blue1Brown

     Worth it: Grid lines bending and stretching on screen: the clearest anyone has ever shown what a matrix does. If you watch one video for linear algebra, make it this one.

   • Read

     Mathematics for Machine Learning, Ch 2 (matrices, linear mappings)

     Deisenroth, Faisal, Ong · free

3. Mark "Determinants" as done

   1.3 Determinants

   • Watch

     Essence of Linear Algebra, Ch 6: The determinant

     3Blue1Brown

     Worth it: Areas growing, shrinking and flipping as the matrix moves space. After this the determinant stops being a formula and becomes a picture.

   • Read

     Mathematics for Machine Learning, Ch 4 (determinant and trace)

     Deisenroth, Faisal, Ong · free

4. Mark "Inverse, rank and solving Ax = b" as done

   1.4 Inverse, rank and solving Ax = b

   • Watch

     Essence of Linear Algebra, Ch 7: Inverse matrices, column space and null space

     3Blue1Brown

   • Watch

     18.06 lecture: Elimination with matrices

     Gilbert Strang (MIT)

   • Read

     Mathematics for Machine Learning, Ch 2 (solving linear systems, basis and rank)

     Deisenroth, Faisal, Ong · free

5. Mark "Dot product, projection and orthogonality" as done

   1.5 Dot product, projection and orthogonality

   • Watch

     Essence of Linear Algebra, Ch 9: Dot products and duality

     3Blue1Brown

   • Watch

     18.06 lecture: Projections onto subspaces

     Gilbert Strang (MIT)

   • Read

     Mathematics for Machine Learning, Ch 3 (inner products, orthogonality, projections)

     Deisenroth, Faisal, Ong · free

6. Mark "Eigenvalues and eigenvectors" as done

   1.6 Eigenvalues and eigenvectors

   • Watch

     Essence of Linear Algebra, Ch 14: Eigenvectors and eigenvalues

     3Blue1Brown

     Worth it: Watching vectors get knocked off their lines while the eigenvectors stay put is the moment eigenvectors make sense. Worth it before PCA.

   • Interactive

     Eigenvectors and Eigenvalues, explained visually

     Setosa · drag the vectors yourself

   • Read

     Mathematics for Machine Learning, Ch 4 (eigenvalues, eigendecomposition)

     Deisenroth, Faisal, Ong · free

7. Mark "SVD, and the maths behind PCA" as done

   1.7 SVD, and the maths behind PCA

   • Watch

     Principal Component Analysis (PCA), Step-by-Step

     StatQuest (Josh Starmer)

   • Watch

     Singular Value Decomposition (SVD): Overview

     Steve Brunton

   • Interactive

     Principal Component Analysis, explained visually

     Setosa · drag the points yourself

     Worth it: Drag the data points and watch the principal directions swing round to follow them. PCA clicks in a couple of minutes.

   • Read

     Mathematics for Machine Learning, Ch 4 (SVD) and Ch 10 (PCA)

     Deisenroth, Faisal, Ong · free

8. Mark "Derivatives and the chain rule" as done

   1.8 Derivatives and the chain rule

   • Watch

     Essence of Calculus, Ch 2: The paradox of the derivative

     3Blue1Brown

   • Watch

     Essence of Calculus, Ch 4: Visualizing the chain rule and product rule

     3Blue1Brown

   • Read

     Mathematics for Machine Learning, Ch 5 (differentiation, the chain rule)

     Deisenroth, Faisal, Ong · free

9. Mark "Partial derivatives, gradients and Jacobians" as done

   1.9 Partial derivatives, gradients and Jacobians

   • Watch

     Gradient and directional derivatives

     Khan Academy

   • Read

     Mathematics for Machine Learning, Ch 5 (partial derivatives, gradients, Jacobians)

     Deisenroth, Faisal, Ong · free

   • Read

     The Matrix Calculus You Need for Deep Learning

     Terence Parr and Jeremy Howard · free

10. Mark "Gradient descent and the learning rate" as done

    1.10 Gradient descent and the learning rate

    • Watch

      Neural Networks, Ch 2: Gradient descent, how neural networks learn

      3Blue1Brown

    • Watch

      Gradient Descent, Step-by-Step

      StatQuest (Josh Starmer)

    • Read

      Mathematics for Machine Learning, Ch 7 (optimisation with gradient descent)

      Deisenroth, Faisal, Ong · free

11. Mark "Random variables, distributions, expectation and variance" as done

    1.11 Random variables, distributions, expectation and variance

    • Watch

      The Normal Distribution, Clearly Explained

      StatQuest (Josh Starmer)

    • Interactive

      Seeing Theory: Probability Distributions

      Brown University · interactive

      Worth it: Roll dice, draw samples and watch histograms and averages settle, live. Probability you can play with instead of only read.

    • Read

      Mathematics for Machine Learning, Ch 6 (probability and distributions)

      Deisenroth, Faisal, Ong · free

12. Mark "Conditional probability and Bayes’ rule" as done

    1.12 Conditional probability and Bayes’ rule

    • Watch

      Bayes' theorem, the geometry of changing beliefs

      3Blue1Brown

      Worth it: Shows Bayes' rule as areas of a square and as counts of people. After it, the classic medical-test puzzle feels obvious instead of strange.

    • Watch

      The medical test paradox

      3Blue1Brown

    • Interactive

      Conditional probability, explained visually

      Setosa · interactive

    • Read

      Mathematics for Machine Learning, Ch 6 (sum rule, product rule, Bayes' theorem)

      Deisenroth, Faisal, Ong · free

13. Mark "Maximum likelihood estimation" as done

    1.13 Maximum likelihood estimation

    • Watch

      Maximum Likelihood, clearly explained

      StatQuest (Josh Starmer)

    • Read

      Mathematics for Machine Learning, Ch 8 (parameter estimation: MLE and MAP)

      Deisenroth, Faisal, Ong · free

    • Read

      Understanding Deep Learning, Ch 5 (loss functions from maximum likelihood)

      Simon Prince · free

14. Mark "Statistics: sampling, confidence intervals, hypothesis tests" as done

    1.14 Statistics: sampling, confidence intervals, hypothesis tests

    • Watch

      Confidence Intervals, Clearly Explained

      StatQuest (Josh Starmer)

    • Watch

      p-values: What they are and how to interpret them

      StatQuest (Josh Starmer)

    • Watch

      But what is the Central Limit Theorem?

      3Blue1Brown

    • Interactive

      Seeing Theory: Frequentist Inference

      Brown University · interactive

15. Mark "Entropy, cross-entropy and KL divergence" as done

    1.15 Entropy, cross-entropy and KL divergence

    • Watch

      Entropy (for data science) Clearly Explained

      StatQuest (Josh Starmer)

    • Watch

      A Short Introduction to Entropy, Cross-Entropy and KL-Divergence

      Aurélien Géron

    • Read

      Visual Information Theory

      Chris Olah · illustrated

      Worth it: Pictures that turn entropy, cross-entropy and KL into areas you can see. The best explanation of why the formulas look the way they do.

16. Mark "Convex optimisation: constraints, Lagrange multipliers and duality" as done

    1.16 Convex optimisation: constraints, Lagrange multipliers and duality

    • Watch

      EE364A Convex Optimization, lecture 1

      Stephen Boyd (Stanford)

    • Read

      Mathematics for Machine Learning, Ch 7 (Lagrange multipliers, convex optimisation)

      Deisenroth, Faisal, Ong · free

    • Read

      Convex Optimization, Ch 2–5

      Boyd and Vandenberghe · free