My new (2018) Deep Learning Tutorial Series: Some theory on Batch Normalization

I will continue to update my Deep Learning notes, and it will be more frequent in 2018 (so please stay-tuned!): Also I have changed from latex/MATLAB to jupyter. So I can share with you all my Python code on Github.

This current tutorial is titled,

From "Why Batch Normalization helps gradient descent faster" To "some fundamentals of multi-variable calculus"

we start with Batch Normalisation, then we discuss the followings:

• What is Data Whitening
• Why BN(or Data Whitening in general) brings symmetric quadratic error surface which helps faster gradient descent

Some multivariate calculus fundamentals on:

• Why gradient vector is orthogonal to level curve
• Some theory on steepest descent direction