Learning track
Neural Network Fundamentals
Learn how neural networks represent functions, measure error, propagate gradients, and improve through optimization.
foundations · 9 available lessons
Neural Network Fundamentals
Learn how neural networks represent functions, measure error, propagate gradients, and improve through optimization.
- The perceptron
- Multilayer perceptrons and non-linearity
- Activation functions: ReLU, GELU, and SwiGLU
- Loss functions and cross-entropy
- Backpropagation, visualized
- Gradient descent and the loss landscape
- Optimizers: SGD to Momentum to Adam to AdamW
- Initialization, normalization, and residuals
- Overfitting, regularization, and the bitter lesson