​A Beginner's Guide to Machine Learning

Machine-Learning tutorial · PySpark.in

1. The Problem We Are Solving

We know that animals can learn patterns from birth — baby ducklings follow the first moving thing they see. We also know that humans are experts at spotting hidden relationships in data. Now we ask the most important question in machine learning:

Given examples of data with correct answers, can we teach a machine to find the hidden pattern and use it to make predictions on new data?

The answer is elegant and comes in two pieces. First, give the machine labeled examples — input data with correct outputs. Second, let the machine find the mathematical relationship between inputs and outputs. The total collection is called supervised learning.

Complete idea: Labeled Data + Pattern Finding Algorithm = Machine Learning

2. Why This Story Matters

The logic is simple. If baby ducklings can imprint on patterns within minutes of hatching, and if humans can spot that y = x 1 + 2x 2 from a small table, then:

Pattern detection (biology) + Mathematical rules (computation) + Iterative improvement (learning) = Artificial Intelligence

So every system of the form "data + pattern finder" is indeed a learning machine. Conversely, if any machine can learn from examples, then it must have both components — examples with known answers and a way to discover the rule connecting them. Every modern AI system, from ChatGPT to self-driving cars, is captured by our formula. The complete picture is exactly the right-hand side shift from fixed programming to adaptive learning: we take the computer and add the ability to find rules from data instead of being told the rules by a programmer.

Historical picture: Konrad Lorenz's ducklings (1935) learned their "mother" by detecting the pattern "first moving thing." The complete solution to machine learning is the same shape — same pattern detection — but shifted so that silicon circuits replace neurons, and mathematical optimization replaces instinct. Scientists call this supervised learning or regression.

3. Finding the Pattern: A Simple Example

To find how machines learn, we start with the simplest possible pattern. The book gives us a beautiful example. Setting the math to its most basic form gives one particular relationship without any complex algorithms.

Let us work through the full story from Anil Ananthaswamy's book. Take this table of data:

Each row in the table is a triplet of values for variables x 1 , x 2 , and y . There's a simple pattern hidden in this data: In each row, the value of y is related to the corresponding values of x 1 and x 2 .

With a pencil, paper, and a little effort one can figure out that y equals x 1 plus two times x 2 :

y = x 1 + 2x 2

Let's verify:

• Row 1: 4 + 2×2 = 8 ✓
• Row 2: 1 + 2×2 = 5 ✓
• Row 3: 0 + 2×5 = 10 ✓
• Row 4: 2 + 2×1 = 4 ✓

Left: The data table with the hidden pattern revealed. Right: Visualizing the linear relationship — each data point sits perfectly on the plane y = x 1 + 2x 2

4. The Complete Solution: From Pattern to Prediction

We already found the pattern. Now we give it a name and understand what it means for machine learning.

A small point about notation: We are going to dispense with the multiplication sign ("×") between two variables or between a constant and a variable. For example, we'll write:

2xx 2 as 2x2_ and x 1xx 2 as x1 x2 _

Ideally, we should write 2x2_ as 2x₂ and x1 x2 _ as x 1x 2 , with the variables subscripted. But we'll dispense with the subscripts, too, unless it becomes absolutely necessary to use them. This method helps keep our text less cluttered and easy on the eye.

Getting back to our equation y = x1 + 2x2, more generally, we can write this as:

y = w1x1 + w2x2, where w1 = 1 and w2 = 2

To be clear, we have found one of the many possible relationships between y and x1 and x2. There can be others. And indeed, for this example, there are, but we don't need to worry about them for our purposes here. Finding patterns is nowhere near as simple as this example is suggesting, but it gets us going.

We identified what's called a linear relationship between y, on the one hand, and x1 and x2, on the other. ("Linear" means that y depends only on x1 and x2, and not on x1 or x2 raised to some power, or on any product of x1 and x2.) Also, I'm using the words "equation" and "relationship" interchangeably here.

The relationship between y, x1, and x2 is defined by the constants w1 and w2. These constants are called the coefficients, or weights, of the linear equation connecting y to x1 and x2.

Left: Biological pattern learning — ducklings imprint on the first moving object. Right: Machine pattern learning — the perceptron finds weights from labeled data. Both systems learn by detecting patterns.

5. The Four Cases: What Makes Learning Hard?

Given a set of n weights, there are important concepts to understand. The number of weights determines both the complexity of the problem and the difficulty of finding them.

Case 1: 2 Weights — Easy to Spot

With just two weights (w1, w2) and a small table, a human can find the pattern by inspection. We just did this!

Example: y = x1 + 2x2 (w1=1, w2=2)

Case 2: 9 Weights — Hard for Humans

With nine weights, you'd be hard-pressed to extract the values of w1 to w9 just by visually inspecting the data and doing some mental arithmetic.

Example: y = w1x1 + w2x2 + w3x3 + ... + w9x9

Case 3: n Weights — Impossible Without Algorithms

Or, more generally, for a set of n weights, and using formal mathematical notation:

y = w1x1 + w2x2 + w3x3 + ... + wnxn =

The expression on the right, using the sigma notation, is shorthand for summing all wi xi , where i takes on values from 1 to n.

In the case of 9 inputs, you'd be hard-pressed to extract the values of w1 to w9 just by visually inspecting the data and doing some mental arithmetic. That's where learning comes in. If there's a way to algorithmically figure out the weights, then the algorithm is "learning" the weights. But what's the point of doing that?

Case 4: Real-World Problems — Millions of Weights

Modern neural networks have millions or even billions of weights. No human could ever find these by hand. We need powerful learning algorithms running on fast computers.

Summary: The number of weights n tells you the complexity of the pattern. When n is small, humans can find weights by inspection. When n is large, we need machine learning algorithms. This is the master key to understanding why machines must learn.

6. Reading the Solution: Supervised Learning

The concept of supervised learning makes everything explicit. After understanding the pattern-finding process, the complete learning pipeline takes a clean form that can be described simply.

Well, once you have learned the weights — say, w1 and w2 in our simple, toy example — then given some value of x1 and x2 that wasn't in our initial dataset, we can calculate the value of y. Say, x1 = 5 and x2 = 2. Plug these values into the equation y = x1 + 2x2 and you get a value of y = 9.

What's all this got to do with real life? Take a very simple, practical, and some would say utterly boring problem. Let's say x1 represents the number of bedrooms in a house, and x2 represents the total square footage, and y represents the price of the house. Let's assume that there exists a linear relationship between (x1, x2) and y. Then, by learning the weights of the linear equation from some existing data about houses and their prices, we have essentially built a very simple model with which to predict the price of a house, given the number of bedrooms and the square footage.

The complete supervised learning pipeline: Training data → Learning algorithm finds weights → Trained model → New data → Prediction.

The above example — a teeny, tiny baby step, really — is the beginning of machine learning. What we just did is a simplistic form of something called supervised learning. We were given samples of data that had hidden in them some correlation between a set of inputs and a set of outputs. Such data are said to be annotated, or labeled; they are also called the training data. Each input (x1, x2, ..., xn) has a label y attached to it. So, in our earlier numerical table, the pair of numbers (4, 2) is labeled with y = 8, the pair (1, 2) with 5, and so on. We figured out the correlation. Once it is learned, we can use it to make predictions about new inputs that weren't part of the training data.

Also, we did a very particular kind of problem solving called regression, where given some independent variables (x1, x2), we built a model (or equation) to predict the value of a dependent variable (y). There are many other types of models we could have built, and we'll come to them in due course.

In this case, the correlation, or pattern, was so simple that we needed only a small amount of labeled data. But modern ML requires orders of magnitude more — and the availability of such data has been one of the factors fueling the AI revolution. (The ducklings, for their part, likely indulge in a more sophisticated form of learning. No parent duck sits around labeling the data for its ducklings, and yet the babies learn. How do they do it? Spoiler alert: We don't know, but maybe by understanding why machines learn, we can one day fully understand how ducklings and, indeed, humans learn.)

To solve a supervised learning problem: (1) collect labeled training data, (2) choose a model form (like y = w1x1 + w2x2), (3) use a learning algorithm to find the weights, (4) verify on test data, (5) use the trained model to predict on new data.

Python Implementation: From Scratch

Below is a complete, zero-library implementation that finds the pattern in data — just like we did mentally, but now the computer does it. This is the simplest possible "machine learning" algorithm: finding weights by brute-force search.

```

python

# ============================================

# BLOG 1: FINDING PATTERNS IN DATA

# Zero-library implementation of pattern finding

# ============================================

def find_linear_pattern(data):

"""

Given data points (x1, x2, y), find weights w1, w2 such that

y = w1*x1 + w2*x2

This is a brute-force search - not efficient, but easy to understand!

"""

print("=" * 55)

print("FINDING THE PATTERN: y = w1*x1 + w2*x2")

print("=" * 55)

print("\nTraining Data:")

print("-" * 30)

print(f"{'x1':>5} {'x2':>5} {'y':>5}")

print("-" * 30)

for x1, x2, y in data:

print(f"{x1:>5} {x2:>5} {y:>5}")

# Try different combinations of w1 and w2

# In real life, we'd use calculus (gradient descent)

# Here we do brute force to keep it simple!

print("\nSearching for weights...")

print("(Trying w1 from -5 to 5, w2 from -5 to 5)")

best_w1, best_w2 = 0, 0

best_error = float('inf')

# Try all combinations with step 0.5

for w1 in [i * 0.5 for i in range(-10, 11)]:

for w2 in [i * 0.5 for i in range(-10, 11)]:

# Calculate total error for this combination

total_error = 0

for x1, x2, y in data:

predicted = w1 * x1 + w2 * x2

error = (predicted - y) ** 2 # Squared error

total_error += error

# Keep track of the best weights

if total_error < best_error:

best_error = total_error

best_w1 = w1

best_w2 = w2

print(f"\n{'='*55}")

print("PATTERN FOUND!")

print(f"{'='*55}")

print(f"Best weights: w1 = {best_w1}, w2 = {best_w2}")

print(f"Total error: {best_error}")

print(f"\n>>> The pattern is: y = {best_w1}*x1 + {best_w2}*x2")

# Verify on training data

print(f"\n{'='*55}")

print("VERIFICATION ON TRAINING DATA:")

print(f"{'='*55}")

print(f"{'x1':>5} {'x2':>5} {'Actual y':>10} {'Predicted':>10} {'Match?':>8}")

print("-" * 55)

all_correct = True

for x1, x2, y in data:

predicted = best_w1 * x1 + best_w2 * x2

match = "YES" if abs(predicted - y) < 0.001 else "NO"

if match == "NO":

all_correct = False

print(f"{x1:>5} {x2:>5} {y:>10} {predicted:>10.1f} {match:>8}")

return best_w1, best_w2, all_correct

def predict(w1, w2, new_data):

"""

Use learned weights to predict on new data

"""

print(f"\n{'='*55}")

print("PREDICTIONS ON NEW DATA:")

print(f"{'='*55}")

print(f"Using: y = {w1}*x1 + {w2}*x2")

print(f"\n{'x1':>5} {'x2':>5} {'Predicted y':>12}")

print("-" * 35)

for x1, x2 in new_data:

y_pred = w1 * x1 + w2 * x2

print(f"{x1:>5} {x2:>5} {y_pred:>12.1f}")

return [w1 * x1 + w2 * x2 for x1, x2 in new_data]

# ============================================

# RUN THE EXAMPLE FROM THE BOOK

# ============================================

# Training data from Ananthaswamy's book (page 9)

training_data = [

(4, 2, 8), # 4 + 2*2 = 8

(1, 2, 5), # 1 + 2*2 = 5

(0, 5, 10), # 0 + 2*5 = 10

(2, 1, 4), # 2 + 2*1 = 4

]

# Find the pattern

w1, w2, perfect = find_linear_pattern(training_data)

# New data to predict on

new_houses = [

(5, 2), # 5 bedrooms, 2000 sq.ft → expected: 5 + 2*2 = 9

(3, 4), # 3 bedrooms, 4000 sq.ft → expected: 3 + 2*4 = 11

(1, 1), # 1 bedroom, 1000 sq.ft → expected: 1 + 2*1 = 3

]

predictions = predict(w1, w2, new_houses)

print(f"\n{'='*55}")

print("WHAT DID WE JUST DO?")

print(f"{'='*55}")

print("""

1. We gave the computer LABELED EXAMPLES (input + correct output)

2. The computer TRIED MANY COMBINATIONS of weights

3. It FOUND THE BEST weights (w1=1, w2=2)

4. Now it can PREDICT on new houses it has never seen!

This is SUPERVISED LEARNING in its simplest form.

""")

```

The Four Cases: What Makes Learning Hard?

Table

Complete solution = labeled training data + learning algorithm that finds weights + trained model that predicts on new data. The pattern complexity comes from the number of weights. The accuracy comes from the quality and quantity of training data.

Why This Was Revolutionary

It may seem implausible, but this first step we took using a laughably simple example of supervised learning sets us on a path toward understanding modern deep neural networks — one step at a time, of course (with small, gentle, and occasionally maybe not so gentle dollops of vectors, matrices, linear algebra, calculus, probability and statistics, and optimization theory served, as needed, along the way).

Rosenblatt's perceptron, which we briefly encountered in the prologue, was for its time an astonishing example of one such learning algorithm. And because it was modeled on how neuroscientists thought human neurons worked, it came imbued with mystique and the promise that, one day, perceptrons would indeed make good on the promise of AI.

While today's AI is far from being able to perform such tasks with the ease and efficiency of ducklings, it does have something in common with the ducklings, and that's the ability to pick out and learn about patterns in data. When Frank Rosenblatt invented the perceptron in the late 1950s, one reason it made such a splash was because it was the first formidable "brain-inspired" algorithm that could learn about patterns in data simply by examining the data. Most important, given certain assumptions about the data, researchers proved that Rosenblatt's perceptron will always find the pattern hidden in the data in a finite amount of time; or, put differently, the perceptron will converge upon a solution without fail. Such certainties in computing are like gold dust. No wonder the perceptron learning algorithm created such a fuss.

Try It Yourself!

Exercise 1: Can you find the pattern in this data?

x1 x2 y

2 3 8

1 1 3

0 2 4

3 0 6

*(Hint: y = ?*x1 + ?x2)

Exercise 2: What if the pattern is NOT linear? Try:

x1 y

1 1

2 4

3 9

4 16

(Hint: y = x1 squared!)

Exercise 3: The book mentions the Mark I Perceptron had 400 inputs. If each pixel is one x value, how many weights would a simple linear model need?

References & Further Reading

Anil Ananthaswamy. Why Machines Learn: The Elegant Math Behind Modern AI. Dutton, 2024.

More Machine-Learning tutorials

• Introduction to Machine Learning
• Introduction to Deep Learning
• The First Artificial Neuron and Learning from Mistakes
• ​Patterns
• ​ Understanding Vectors, the Building Blocks of AI
• How Vectors, Dot Products, and Hyperplanes Power the Perceptron

All tutorials · Try the free PySpark compiler · Practice challenges