Introduction to Deep Learning

Machine-Learning tutorial · PySpark.in

Introduction to Deep Learning

Beginner's Guide to AI & Deep Learning | Part 2 of 2

Prerequisites:

• Introduction to Machine Learning Blog: read Part 1 first

• Biological neuron: how a brain cell receives and transmits signals

• Piecewise function: a function defined differently on different intervals

• Boolean logic: AND, OR, NOT gates and binary (0/1) outputs

• Hyperplane: a flat surface that divides a space into two regions

1.1 How We Got the First Artificial Neuron

Every revolution has an origin story. To understand deep learning, we need to go back to 1943, when two scientists — Warren McCulloch and Walter Pitts — asked a bold question: could the behaviour of a biological brain cell be described with mathematics?

A biological neuron works like this: it receives electrical signals through its branching dendrites, processes those signals in its cell body, and if the total incoming signal is strong enough, it fires an output signal down its axon to neighbouring neurons. The key idea is the threshold — the neuron fires only when its total input crosses a certain level.

🖼 Figure 1.1 — Diagram of a biological neuron: dendrites → cell body → axon → output

McCulloch and Pitts used this biological behaviour to create a simple mathematical model — the first artificial neuron. Their model takes multiple binary inputs (0 or 1), adds them up, and fires an output of 1 only if the sum meets or exceeds a threshold θ. Otherwise, the output is 0.

🖼 Figure 1.2 — McCulloch-Pitts artificial neuron: inputs x₁...xₙ → sum g(x) → threshold function f(z)

g(x) = x1 + x2 + x3 + ... + xn (inputs are binary: 0 or 1)

z = g(x)

f(z) = 0 if z < threshold θ

f(z) = 1 if z >= threshold θ

What is remarkable about this model is how naturally it maps onto logical reasoning. By simply changing the threshold, the same artificial neuron can implement different logic gates:

• AND gate: set the threshold equal to the total number of inputs. The neuron fires only when every single input is active — exactly the behaviour of an AND gate.

• OR gate: lower the threshold to 1. Now the neuron fires whenever at least one input is active — exactly the OR gate behaviour.

By combining many such neurons together, any boolean logic function can in principle be computed. This was profound: it showed that logical reasoning could emerge from a very simple mathematical unit.

What McCulloch and Pitts gave us was a machine that could compute, but not learn. The threshold value θ had to be set by hand — there was no mechanism for the neuron to figure out the right threshold from data. That crucial limitation would soon be solved.

1.2 The First Learning Machine: The Perceptron

In the late 1950s, Frank Rosenblatt built the Mark I Perceptron — a physical device with a camera that produced 20×20 pixel images, which it used to recognise letters of the alphabet. Crucially, this device could learn from its mistakes. It was zapped — given a negative signal — whenever it classified a letter incorrectly, and it adjusted its internal parameters in response. This was machine learning in hardware.

🖼 Figure 1.3 — The Perceptron: inputs x₁...xₙ with weights w₁...wₙ → weighted sum g(x) + bias b → output y

The perceptron improved on the McCulloch-Pitts neuron in two important ways. First, inputs no longer had to be binary — the perceptron could accept any numerical values. Second, each input was multiplied by a learnable weight before being summed:

g(x) = b + w1*x1 + w2*x2 + ... + wn*xn

z = g(x)

y = f(z) = -1 if z <= 0

y = f(z) = +1 if z > 0

The weights wᵢ and the bias term b are all learnable parameters. The output y is either +1 or -1 — a binary classification. The perceptron's learning algorithm automatically adjusts the weights until the output is correct. This is exactly the same idea as finding the weights W1 and W2 in the machine learning blog, but now done automatically from data, without any human guidance.

1.3 Classification Using the Perceptron

To see the perceptron in action, consider a concrete classification task. We want to classify people as obese (y = +1) or non-obese (y = -1) based on their body weight (x₁) and height (x₂). We have a dataset of one hundred labelled examples — each person's weight, height, and their obesity label.

The perceptron needs to find values for w₁, w₂, and the bias b such that:

g(x) = b + w1 * weight + w2 * height

Once the perceptron has learned the correct weights, it can take any new person's weight and height — someone not in the original dataset — and predict whether they are obese or not.

🖼 Figure 1.4 — Scatter plot: obese vs non-obese data points with a separating line (hyperplane) drawn by the perceptron

Geometrically, what the perceptron is doing is finding a hyperplane — in this 2D case, simply a straight line — that separates the two categories of data points. On one side of the line lie the obese data points; on the other side lie the non-obese ones. This task is called binary classification, and it is one of the most fundamental problems in machine learning.

Important caveat: the perceptron assumes that such a separating hyperplane exists — that the two categories are linearly separable. If no straight line (or flat plane) can separate the data, the perceptron will fail to converge. This limitation motivated much of the later work that led to deep learning.

1.4 How Does a Perceptron Learn?

The learning process of the perceptron is elegant in its simplicity: start with zero knowledge, make a prediction, check whether it is right, and correct the weights whenever a mistake is made. Repeat until no mistakes remain.

Step 1 — Start with Zero Knowledge

All weights (w₁, w₂) and the bias b are initialised to zero. This means the perceptron begins with no preference for any direction — its initial decision boundary is essentially arbitrary and will almost certainly be wrong.

Step 2 — Check Each Data Point and Correct Mistakes

For each data point in the training dataset, the perceptron computes g(x) using the current weights. It then checks whether the prediction agrees with the true label:

• label × g(x) > 0: the point is classified correctly. No change needed.

• label × g(x) ≤ 0: the point is misclassified. Update the weights and bias to move the decision boundary slightly toward the correct side.

Step 3 — Repeat Until No Mistakes Remain

After checking all data points once (one full pass, or epoch), the perceptron checks whether any updates were made. If at least one update happened, the model is still making mistakes — repeat Step 2. If no updates happened during the full pass, all points are now correctly classified and learning stops. The perceptron has found suitable values for w₁, w₂, and b.

Graphically, this learning process looks like a line slowly rotating and shifting across the scatter plot. At first the line is in the wrong position and misclassifies many points. Each mistake nudges the line a little. After several passes, the line settles into the correct position, cleanly separating the two classes.

🖼 Figure 1.5 — Iteration 1: initial hyperplane, many misclassifications

🖼 Figure 1.6 — Iteration 2: hyperplane begins to move toward correct position

🖼 Figure 1.7 — Iteration 3: hyperplane converging

🖼 Figure 1.8 — Final iteration: hyperplane correctly separates both classes

1.5 From Perceptrons to Deep Learning

The perceptron may look deceptively simple — a handful of inputs, some weights, and a decision rule. But this simple idea represented a major turning point in human understanding of intelligence. For the first time, a machine demonstrated that it could learn from data and improve its own behaviour through experience, rather than following only fixed, hand-written rules.

This principle — learn from examples, correct mistakes, improve over time — became the foundation for everything that came after in artificial intelligence.

Stacking Perceptrons: The Neural Network

Deep learning takes this same principle and scales it dramatically. Instead of a single perceptron, we connect many perceptrons together in multiple layers, forming what we now call a neural network. The word deep simply refers to having many layers — sometimes hundreds.

Each layer in a deep network learns a slightly more abstract representation of the data than the layer before it. In an image recognition system, for example, the first layer might learn to detect edges, the second layer might combine edges into shapes, the third layer might combine shapes into objects, and so on — each layer building on the patterns discovered by the previous one.

• A perceptron learns a single linear decision boundary — one straight line or hyperplane.

• Deep learning stacks thousands or millions of such learning units together, allowing machines to recognise images, understand natural language, generate text, and make complex decisions in ways that far exceed what any single perceptron could do.

The perceptron did not just solve a binary classification problem. It opened an entirely new frontier of knowledge — showing that learning itself could be modelled mathematically. Deep learning is the modern continuation of that same idea, scaled up with more data, more computation, and deeper networks, but still rooted in the same core learning principle that Rosenblatt demonstrated with his Mark I machine in the late 1950s.

From four rows of data (Blog 1) to a 20×20 pixel camera learning the alphabet (Blog 2) to modern neural networks learning to write poetry and diagnose cancer — the journey is one of scale and depth, but the underlying idea has never changed: find the weights, correct the mistakes, and repeat.

1.6 A Preview: Implementing a Perceptron

The Perceptron algorithm can be implemented in just a few lines of Python using the scikit-learn library. The library handles the weight updates and convergence checking automatically, allowing you to focus on understanding the inputs and outputs rather than the mechanics of the learning loop.

```

from sklearn.linear_model import Perceptron

import numpy as np

# Training data: [weight, height] for each person

X = np.array([[80, 160], [90, 155], [60, 170], [70, 175]])

y = np.array([1, 1, -1, -1]) # 1 = obese, -1 = non-obese

# Create and train the perceptron

clf = Perceptron(max_iter=1000)

clf.fit(X, y)

# Predict a new person: weight=75, height=165

print(clf.predict([[75, 165]]))

```

Under the hood, the Perceptron class in scikit-learn is executing exactly the three-step loop we described in Section 1.4 — initialise weights to zero, check each point, update on mistakes, repeat. The implementation simply automates what we described conceptually, letting you apply it to real datasets with minimal effort.

In the coming blogs, we will extend this idea to multi-layer neural networks, explore how they handle non-linear problems that a single perceptron cannot solve, and understand the backpropagation algorithm that makes learning possible across many layers.

References & Further Reading

Ananthaswamy, Anil. Why Machines Learn. Dutton, 2018.

Rosenblatt, Frank. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological Review, 1958.

Nielsen, Michael. Neural Networks and Deep Learning. neuralnetworksanddeeplearning.com, 2015.

End of Blog 2 — AI & Deep Learning Series

Introduction to Deep Learning