🧠

Simple Perceptron

The perceptron is the fundamental building block of neural networks. Watch it learn to classify data points by adjusting its weights and bias.

The Perceptron

A perceptron is the simplest type of artificial neuron. It takes inputs, multiplies them by weights, adds a bias, and produces an output through an activation function.

output = activate(w₁·x + w₂·y + b)

The perceptron learns by adjusting weights and bias to minimize prediction errors.

Visualization

Class 0 (Below line)
Class 1 (Above line)
Misclassified

Parameters

Training Statistics

Epoch
0
Accuracy
0.0%
Data Points
0
Status
Ready

How the Perceptron Learns

1
Forward Pass: Calculate output = activate(w₁·x + w₂·y + b)
2
Calculate Error: error = actual_label - predicted_label
3
Update Weights: w₁ = w₁ + learning_rate × error × x
4
Repeat: Continue until accuracy is satisfactory

Understanding the Perceptron

What is a Perceptron?

A perceptron is a simple artificial neuron that takes multiple inputs, applies weights to them, adds a bias, and produces a binary output (0 or 1). It was invented by Frank Rosenblatt in 1957 and is the foundation of modern neural networks.

How Does It Learn?

The perceptron uses supervised learning. For each training example:

  1. It makes a prediction based on current weights
  2. Compares the prediction with the actual label
  3. Calculates the error (difference between prediction and actual)
  4. Updates weights proportionally to the error and learning rate

Key Components

Weights (w₁, w₂): Control how much each input influences the output

Bias (b): Shifts the decision boundary, allowing the model to fit better

Activation Function: Converts the weighted sum to a binary output (step function)

Learning Rate: Controls how much weights change with each update

Mathematical Formulation

Forward Pass

z = w₁ × x + w₂ × y + b

output = activate(z) = {1 if z ≥ 0, 0 otherwise}

Weight Update Rule

error = actual_label - predicted_label

w₁ = w₁ + α × error × x

w₂ = w₂ + α × error × y

b = b + α × error

(where α is the learning rate)

Limitations of the Perceptron

⚠️
Only Linearly Separable Data: A single perceptron can only classify data that can be separated by a straight line (or hyperplane in higher dimensions).
⚠️
Cannot Solve XOR Problem: The famous XOR problem cannot be solved by a single perceptron, leading to the development of multi-layer networks.
✅
Solution - Multi-Layer Perceptrons: By stacking multiple layers of perceptrons with non-linear activations, we can solve complex non-linear problems.

Historical Significance

1957: Frank Rosenblatt invented the perceptron at the Cornell Aeronautical Laboratory.

1958: The Mark I Perceptron was the first machine to recognize simple shapes and patterns, sparking excitement about artificial intelligence.

1969: Minsky and Papert's book "Perceptrons" highlighted limitations, causing the first "AI Winter."

1980s: The development of backpropagation and multi-layer networks overcame these limitations, leading to modern deep learning.

Experiment Ideas

1.

Try different learning rates and observe how quickly the perceptron converges

2.

Generate new data and see how the perceptron adapts to different distributions

3.

Manually adjust weights to understand how they affect the decision boundary

4.

Watch how the bias parameter shifts the decision line up or down