Building a Simple Perceptron in Python: A Step-by-Step Guide

Perceptrons are the foundation of neural networks and are an excellent starting point for beginners venturing into machine learning and artificial intelligence. In this tutorial, we’ll build a simple perceptron model in Python to understand how it works under the hood. We’ll train it to solve a basic binary classification task.

What is a Perceptron?

A perceptron is a single-layer neural network that maps input features to an output using weights, a bias, and an activation function. It’s one of the simplest forms of artificial neural networks, invented by Frank Rosenblatt in 1958.

The key steps in a perceptron are:

1. Initialization: Assign random weights.
2. Forward Pass: Compute outputs using the activation function.
3. Error Calculation: Measure the difference between actual and predicted outputs.
4. Weight Update: Adjust weights using the error.

Step-by-Step Implementation

Here’s how you can implement a perceptron in Python:

1. Import Libraries

We’ll use numpy for numerical computations.

import numpy as np        

2. Define the Activation Function

We’ll use the sigmoid function, which outputs values between 0 and 1, making it suitable for binary classification tasks.

def sigmoid(x):
    return 1 / (1 + np.exp(-x))


def sigmoid_derivative(x):
    return x * (1 - x)        

The derivative of the sigmoid function helps in updating weights during training.

3. Prepare Training Data

We’ll use a small dataset with binary inputs and outputs.

training_inputs = np.array([[0, 0, 1],
                            [1, 1, 1],
                            [1, 0, 1],
                            [0, 1, 1]])

training_outputs = np.array([[0, 1, 1, 0]]).T        

Each row of training_inputs represents one training example, and the corresponding value in training_outputs is the target output.

4. Initialize Weights

We assign random values to the weights, ensuring they’re initially small.

np.random.seed(1)
synaptic_weights = 2 * np.random.random((3, 1)) - 1

print("Random starting synaptic weights...")
print(synaptic_weights)        

5. Train the Perceptron

We’ll iterate through the training data and adjust the weights to minimize errors.

for iterations in range(100000):
    # Forward pass
    input_layer = training_inputs
    outputs = sigmoid(np.dot(input_layer, synaptic_weights))

    # Calculate error
    error = training_outputs - outputs

    # Adjust weights
    adjustments = error * sigmoid_derivative(outputs)
    synaptic_weights += np.dot(input_layer.T, adjustments)        

6. Display Results

After training, we’ll print the updated weights and final outputs.

print("Synaptic weights after training")
print(synaptic_weights)

print("Outputs after training: ")
print(outputs)        

Full Code

Here’s the complete implementation:

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1 - x)

training_inputs = np.array([[0, 0, 1],
                            [1, 1, 1],
                            [1, 0, 1],
                            [0, 1, 1]])

training_outputs = np.array([[0, 1, 1, 0]]).T

np.random.seed(1)

synaptic_weights = 2 * np.random.random((3, 1)) - 1
 
print("Random starting synaptic weights...")

print(synaptic_weights)

for iterations in range(100000):
    input_layer = training_inputs
    outputs = sigmoid(np.dot(input_layer, synaptic_weights))
    error = training_outputs - outputs
    adjustments = error * sigmoid_derivative(outputs)
    synaptic_weights += np.dot(input_layer.T, adjustments)

print("Synaptic weights after training")
print(synaptic_weights)
print("Outputs after training: ")
print(outputs)        

Explanation of the Code

1. Random Weight Initialization:

• We initialize the weights randomly in the range [-1, 1].

2. Forward Pass:

• Compute the dot product of inputs and weights.
• Apply the sigmoid function to normalize the result.

3. Error Calculation:

• Subtract the predicted output from the actual output.

4. Backpropagation:

• Use the derivative of the sigmoid function to compute adjustments.
• Update the weights to reduce the error.

Outputs

1. Initial Weights: Randomly generated weights before training.
2. Final Weights: Updated weights after training for 100,000 iterations.
3. Predicted Outputs: The perceptron’s predictions for the training inputs.

Extending the Model

This perceptron works for binary classification with a single layer. To handle more complex datasets, you can:

• Add more layers (deep learning).
• Use other activation functions (ReLU, tanh).
• Incorporate advanced optimization techniques like gradient descent.

Conclusion

Congratulations! You’ve just built and trained a simple neural network from scratch. This foundational understanding will help you grasp more advanced neural networks and deep learning concepts. Experiment with different datasets and parameters to see how the perceptron performs. Happy coding!