Understanding Idea behind Full Backpropogation

Thu, 02 Jul 2026 18:14:45 +0000

Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.

In the previous article, we learned how to calculate the gradient of the last bias in a neural network.

Now we will explore how gradients flow through the entire network and how to calculate the weights of previous layers.

How to calculate weights

Now we will calculate weights of the previous layer

The challenge is that the loss function does not directly depend on these earlier weights.

For example, consider a weight (w1 and w2 are already calculated ) in a hidden layer.

Changing (w3 and w4):

Changes the hidden neuron output.
Changes the output neuron input.
Changes the final prediction.
Changes the loss.

So there is an indirect relationship between the weight and the loss.

This is exactly why we need the chain rule.

Conclusion

Similar to previous calculation we should also calculate for all weights and biases using chain rule and gradient descent.

Any feedback or contributors are welcome! It’s online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

How to calculate weights using gradient descent

Thu, 02 Jul 2026 02:57:30 +0000

In the previous article, I explained the requirements for finding the weights of the last layer. Now let's see how we actually assign and optimize those weights using gradient descent.

Selecting Random Weights From a Normal Distribution

First, we assign random numbers (drawn from a normal distribution) to the weights w3 and w4.

Then we sum up both results along with the bias b3 = 0:

predicted = (output of top neuron × w3) + (output of bottom neuron × w4) + b3

This gives us our initial prediction, and we can plot the final graph:

With these initial random weights, the SSR (Sum of Squared Residuals) is calculated again to measure how far off our predictions are.

Gradient Descent Algorithm For Optimal Values

Now we need to find the derivative of SSR with respect to b3 so we can update it.

Recall our loss function:

SSR = Σ (observed − predicted)²

And our predicted value is:

predicted = (output of top neuron × w3) + (output of bottom neuron × w4) + b3

This is the same chain rule approach we used for backpropagation when optimizing only b3.

The key insight here is that the products of w3 and w4 with their respective neuron outputs are treated as constants for a single gradient calculation with respect to b3. Since only b3 is the variable in this expression, the derivative simplifies cleanly — just as we saw in the previous articles.

Conclusion

By assigning random weights from a normal distribution and then applying gradient descent with the chain rule, we can iteratively optimize each weight and bias in the network. The same process that worked for b3 alone now extends to w3 and w4 — we just need to carefully apply the chain rule at each step to compute the correct gradients.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Coder Legion - Recent posts tagged weights

Understanding Idea behind Full Backpropogation

How to calculate weights

Conclusion

How to calculate weights using gradient descent

Selecting Random Weights From a Normal Distribution

Gradient Descent Algorithm For Optimal Values

Conclusion