ABOUT CATEGORIES ARCHIVE

Posts tagged with "vanishing gradient"

There are a total of 2 posts.

Vanishing Gradient – Why Deep Learning Struggles with Short‑Term Memory Loss

2026-02-11

Vanishing Gradient – Why Deep Learning Struggles with Short‑Term Memory Loss

Explore why the vanishing gradient problem plagues deep learning, the role of sigmoid and the chain rule, and how ReLU and ResNet elegantly solve it—complete with practical insights for modern neural network design.

#deep learning #backpropagation #vanishing gradient #relu +2
Backpropagation: How AI Traces the Culprit of Errors (feat. the Chain Rule)

2025-12-19

Backpropagation: How AI Traces the Culprit of Errors (feat. the Chain Rule)

Explore how backpropagation lets AI pinpoint error sources using the chain rule, how PyTorch’s Autograd automates gradient calculation, and how the vanishing gradient problem threatens deep networks—and how modern techniques like ReLU and ResNet address it.

#deep learning #ai #backpropagation #chain rule +2