Neural Network Part 2: Forward Propagation Implementation

In our previous article, we set up our neural network to predict food insecurity scores based on rainfall and crop yield data. We covered data preparation feature scaling and implemented our neural network in Python. Now, in this article we'll be focusing on the forward propagation step.

If you haven't read Part 1 yet, I recommend checking it out here to get familiar with data preparation, scaling, and neural network setup.

What is Forward Propagation?

Forward propagation is the process where input data moves through the network layer by layer to produce an output. During this process, the network calculates weighted sums, applies activation functions, and generates predictions.

Why We Need Activation Functions

Activation functions are crucial in neural networks because:

1. Introducing Non-linearity: Without activation functions, our network would only be a series of linear transformations, unable to learn complex patterns. Most real-world problems (like food insecurity prediction) involve non-linear relationships.
2. Feature Learning: Activation functions help neurons learn different features or patterns in the data.
3. Output Transformation: For our regression problem, activation functions help constrain outputs to a meaningful range (between 0 and 1).

The sigmoid function is particularly appropriate for our model because it produces values between 0 and 1, which aligns with our scaled outputs.

It's defined as:

def sigmoid(self, z):
    """
    Compute the sigmoid of z
    """
    return 1 / (1 + np.exp(-z))
        

Forward Propagation Implementation

Now, let's implement the forward propagation in python:

def forward(self, X):
    """
    Forward propagation through the network
    
    Parameters:
    X -- input data, shape (m, 2) where m is the number of examples
    
    Returns:
    yHat -- predictions, shape (m, 1)
    """
    # First layer transformation
    Z2 = np.dot(X, self.W1) + self.b1  # Shape (m, 3)
    
    # Apply activation function to first layer
    A2 = self.sigmoid(Z2)  # Shape (m, 3)
    
    # Second layer transformation
    Z3 = np.dot(A2, self.W2) + self.b2  # Shape (m, 1)
    
    # Apply activation function to second layer
    yHat = self.sigmoid(Z3)  # Shape (m, 1)
    
    return yHat
        

Note that we apply the sigmoid activation function twice - once after the hidden layer computation and once after the output layer computation. This introduces the necessary non-linearity at each step of the network.

The Forward Propagation Process

Let's break down what happens during forward propagation with our food insecurity example:

1. Input Layer: The scaled rainfall and crop yield values enter the network
2. Hidden Layer Computation: Each input is multiplied by weights and summed
3. First Activation: The sigmoid function transforms these sums to values between 0 and 1
4. Output Layer Computation: The hidden layer activations are multiplied by the second set of weights
5. Second Activation: The sigmoid function transforms the final output to a value between 0 and 1

The Complete Neural Network Class in Python

Here's the complete neural network class with forward propagation:

import numpy as np

class NeuralNetwork:
    def __init__(self):
        """
        Initialize neural network parameters
        """
        # Set random seed for reproducibility
        np.random.seed(42)
        
        # Define hyperparameters
        self.inputLayerSize = 2     # Number of input neurons (rainfall and crop yield)
        self.hiddenLayerSize = 3    # Number of hidden neurons
        self.outputLayerSize = 1    # Number of output neurons (food insecurity score)
        
        # Initialize weights randomly with small values to prevent symmetry issues
        self.W1 = np.random.randn(self.inputLayerSize, self.hiddenLayerSize) * 0.01  # Shape (2,3)
        self.W2 = np.random.randn(self.hiddenLayerSize, self.outputLayerSize) * 0.01  # Shape (3,1)
        
        # Initialize bias terms as zeros (they'll be updated during training)
        self.b1 = np.zeros((1, self.hiddenLayerSize))  # Shape (1,3) - Bias for hidden layer
        self.b2 = np.zeros((1, self.outputLayerSize))  # Shape (1,1) - Bias for output layer
    
    def sigmoid(self, z):
        """
        Compute the sigmoid of z
        """
        return 1 / (1 + np.exp(-z))
    
    def forward(self, X):
        """
        Forward propagation through the network
        
        Parameters:
        X -- input data, shape (m, 2) where m is the number of examples
        
        Returns:
        yHat -- predictions, shape (m, 1)
        """
        # First layer transformation
        Z2 = np.dot(X, self.W1) + self.b1  # Shape (m, 3)
        
        # Apply activation function to first layer
        A2 = self.sigmoid(Z2)  # Shape (m, 3)
        
        # Second layer transformation
        Z3 = np.dot(A2, self.W2) + self.b2  # Shape (m, 1)
        
        # Apply activation function to second layer
        yHat = self.sigmoid(Z3)  # Shape (m, 1)
        
        return yHat
        

Testing Our Network

Now let's test our neural network with our scaled data:

# Create our neural network
nn = NeuralNetwork()

# Make predictions using our untrained network
predictions_scaled = nn.forward(X_scaled)
predictions = predictions_scaled * max_y

# Compare predictions with actual values
for i, region in enumerate(['A', 'B', 'C', 'D', 'E', 'F']):
    print(f"Region {region} - Actual: {y[i][0]}, Predicted: {predictions[i][0]:.1f}")
        

And there you have it! A Python class capable of estimating food insecurity scores given rainfall and crop yield data. You may notice that our estimates are quite terrible right now. That's because we have not yet trained our network - the weights are still random values.

These components allow our network to make predictions, albeit untrained ones. In the next part, we'll focus on training our network using backpropagation and gradient descent to adjust the weights and biases, transforming our network from a random predictor into an accurate forecasting tool.

Stay tuned for Part 3, where we'll dive into training our neural network!

Feel free to share your thoughts ,questions or insights in the comments.