Understanding Multiple Input and Output Neural Network

Wed, 15 Jul 2026 13:17:23 +0000

Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.

In the previous article, we discussed how ReLU activation function works. Now let's see how multiple input and multiple output neural networks work.

How Multiple Input and Output Neural Network works

Until now, we worked with a neural network that had a single input and a single output. In real-world problems, we usually have multiple inputs and multiple outputs.

In this article, we will build a neural network with:

2 inputs (input1 and input2)
1 hidden layer with 2 neurons
3 outputs (output1, output2, output3)
ReLU as the activation function

Network Structure

input1 ──┐
         ├──► hidden_neuron1 ──┬──► output1
input2 ──┤                     ├──► output2
         └──► hidden_neuron2 ──┴──► output3

Each input is connected to each hidden neuron, and each hidden neuron is connected to each output neuron.

Forward Pass Equations

All weights and biases are assigned based on normal distribution.
Hidden Layer Calculation
For hidden neuron 1:

x1 = (input1 * w1) + (input2 * w2) + b1
y1 = ReLU(x1) = max(0, x1)

For hidden neuron 2:

x2 = (input1 * w3) + (input2 * w4) + b2
y2 = ReLU(x2) = max(0, x2)

Output Layer Calculation

For output1:

output1 = (y1 * w5) + (y2 * w6) + b3

For output2:

output2 = (y1 * w7) + (y2 * w8) + b4

For output3:

output3 = (y1 * w9) + (y2 * w10) + b5

Example with Numbers

Let's work through a concrete example.

Given inputs:

input1 = 2
input2 = 3

Assume the following initial weights and biases:

Hidden layer weights:
  w1 = 0.5,  w2 = -0.3,  b1 = 0.1
  w3 = -0.4, w4 = 0.8,   b2 = 0.2

Output layer weights:
  w5 = 0.6,  w6 = 0.7,  b3 = 0.1
  w7 = -0.5, w8 = 0.4,  b4 = 0.2
  w9 = 0.3,  w10 = -0.6, b5 = 0.0

Calculate hidden neuron values

Hidden neuron 1:

x1 = (2 * 0.5) + (3 * -0.3) + 0.1
   = 1.0 - 0.9 + 0.1
   = 0.2

y1 = ReLU(0.2) = max(0, 0.2) = 0.2

Hidden neuron 2:

x2 = (2 * -0.4) + (3 * 0.8) + 0.2
   = -0.8 + 2.4 + 0.2
   = 1.8

y2 = ReLU(1.8) = max(0, 1.8) = 1.8

Calculate outputs

Output 1:

output1 = (0.2 * 0.6) + (1.8 * 0.7) + 0.1
        = 0.12 + 1.26 + 0.1
        = 1.48

Output 2:

output2 = (0.2 * -0.5) + (1.8 * 0.4) + 0.2
        = -0.10 + 0.72 + 0.2
        = 0.82

Output 3:

output3 = (0.2 * 0.3) + (1.8 * -0.6) + 0.0
        = 0.06 - 1.08 + 0.0
        = -1.02

Final Results

output1 = 1.48
output2 = 0.82
output3 = -1.02

Why ReLU works here

Notice that x1 = 0.2 and x2 = 1.8 — both are positive, so ReLU passes them through unchanged.

If any x value were negative (say x1 = -0.5), then:

y1 = ReLU(-0.5) = max(0, -0.5) = 0

That neuron would contribute nothing to the outputs, effectively "turning off" and making the network sparse and efficient.

Matrix Representation

You can think of all the weights as a matrix of connections:

Hidden Layer (2x2 weight matrix + 2 biases):

  w1   w2   b1        w3   w4   b2
[ 0.5  -0.3  0.1 ]  [ -0.4  0.8  0.2 ]


Output Layer (3x2 weight matrix + 3 biases):

  w5   w6   b3
[ 0.6  0.7  0.1 ]   → output1

  w7   w8   b4
[-0.5  0.4  0.2 ]   → output2

  w9  w10   b5
[ 0.3 -0.6  0.0 ]   → output3

Each output neuron learns a different combination of the hidden neuron outputs, allowing the network to produce multiple independent predictions.

Conclusion

We now understand how a neural network with 2 inputs and 3 outputs works step by step using the ReLU activation function:

Each hidden neuron receives all inputs, computes a weighted sum plus bias, and applies ReLU.
Each output neuron receives all hidden neuron outputs and computes its own weighted sum plus bias.
The network can produce multiple distinct outputs simultaneously from the same inputs.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

How to calculate weights using gradient descent

Thu, 02 Jul 2026 02:57:30 +0000

In the previous article, I explained the requirements for finding the weights of the last layer. Now let's see how we actually assign and optimize those weights using gradient descent.

Selecting Random Weights From a Normal Distribution

First, we assign random numbers (drawn from a normal distribution) to the weights w3 and w4.

Then we sum up both results along with the bias b3 = 0:

predicted = (output of top neuron × w3) + (output of bottom neuron × w4) + b3

This gives us our initial prediction, and we can plot the final graph:

With these initial random weights, the SSR (Sum of Squared Residuals) is calculated again to measure how far off our predictions are.

Gradient Descent Algorithm For Optimal Values

Now we need to find the derivative of SSR with respect to b3 so we can update it.

Recall our loss function:

SSR = Σ (observed − predicted)²

And our predicted value is:

predicted = (output of top neuron × w3) + (output of bottom neuron × w4) + b3

This is the same chain rule approach we used for backpropagation when optimizing only b3.

The key insight here is that the products of w3 and w4 with their respective neuron outputs are treated as constants for a single gradient calculation with respect to b3. Since only b3 is the variable in this expression, the derivative simplifies cleanly — just as we saw in the previous articles.

Conclusion

By assigning random weights from a normal distribution and then applying gradient descent with the chain rule, we can iteratively optimize each weight and bias in the network. The same process that worked for b3 alone now extends to w3 and w4 — we just need to carefully apply the chain rule at each step to compute the correct gradients.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star git-lrc on GitHub

Coder Legion - Recent posts tagged neuron

Understanding Multiple Input and Output Neural Network

How Multiple Input and Output Neural Network works

Network Structure

Forward Pass Equations

Example with Numbers

Why ReLU works here

Matrix Representation

Conclusion

How to calculate weights using gradient descent

Selecting Random Weights From a Normal Distribution

Gradient Descent Algorithm For Optimal Values

Conclusion