​Patterns

Machine-Learning tutorial · PySpark.in

1. The Problem We Are Solving

We know that machines can learn from data. But before we dive into neurons and algorithms, we need to answer the most basic question:

What exactly is a "pattern," and how do we know when we've found one?

The answer is simple but powerful. A pattern is a rule that connects inputs to outputs — a relationship that lets us predict something we don't know from something we do know. The total collection of finding, testing, and using this rule is what we call learning.

Complete idea: Input + Hidden Rule = Predictable Output

2. Why This Matters for Beginners

The logic is simple. If you can spot that 2, 4, 6, 8 follows the rule "add 2," then you can predict the next number is 10. If a machine can spot that house size and price are connected by a straight line, then it can predict the price of a house it's never seen before.

Human pattern spotting + Mathematical description + Computer power = Machine Learning

So every system that finds rules from examples is indeed a learning machine. The complete picture is exactly the shift from "guessing with your gut" to "calculating with confidence": we take the human ability to see connections and add mathematical precision and computational speed.

Intuitive picture: a pattern is like a secret code hidden in data. Once you crack the code, you can read messages you've never seen before. Mathematicians call this a function or mapping; in machine learning, we call it a model.

3. What is a Pattern? Four Types of Data

To build intuition, let's look at four different situations. Some have patterns. Some don't. Learning to tell the difference is the first skill of a data scientist.

Type 1: NO PATTERN — Random Data (Top-Left)

Look at the red dots scattered everywhere. If I give you x = 5, can you tell me y? No! The points have no relationship. This is random noise.

Without a pattern, prediction is impossible.

Type 2: SIMPLE PATTERN — Linear Relationship (Top-Right)

Now look at the blue dots. They follow a straight line! When x goes up, y goes up in a predictable way. The rule is:

y = 2x + 1

If x = 5, then y = 2(5) + 1 = 11. We can predict!

Type 3: HIDDEN PATTERN — Exponential Growth (Bottom-Left)

These purple dots look crazy at first. The y-values jump from 2 to 4 to 8 to 16... Do you see it? Each y is double the previous one. The rule is:

y = 2 x

Sometimes the pattern is not a straight line. But it's still a pattern!

Type 4: REAL-WORLD PATTERN — House Prices (Bottom-Right)

These orange dots show real house data. They're not perfectly on a line (real life is messy!), but there's a clear trend. Bigger houses cost more. We can draw a "best fit" line through the noise.

Real patterns are usually fuzzy — but still useful!

4. The Complete Solution: The "Spot the Pattern" Game

Let's play a game! I'll show you data. You find the pattern. This is exactly what a machine does — just much faster.

Puzzle 1: Simple Addition

x: 1 2 3 4 5

y: 3 4 5 6 7

Look at the numbers. When x goes up by 1, y goes up by 1. But y is always 2 more than x.

Pattern: y = x + 2

Check: 1+2=3 ✓, 2+2=4 ✓, 3+2=5 ✓, 4+2=6 ✓, 5+2=7 ✓

Puzzle 2: Multiplication

x: 1 2 3 4 5

y: 2 4 6 8 10

When x goes up by 1, y goes up by 2. y is always double x.

Pattern: y = 2x

Check: 2×1=2 ✓, 2×2=4 ✓, 2×3=6 ✓, 2×4=8 ✓, 2×5=10 ✓

Puzzle 3: Squared

x: 1 2 3 4 5

y: 1 4 9 16 25

These numbers grow fast! 1×1=1, 2×2=4, 3×3=9...

Pattern: y = x²

Check: 1²=1 ✓, 2²=4 ✓, 3²=9 ✓, 4²=16 ✓, 5²=25 ✓

Puzzle 4: Two Inputs!

x1: 1 2 3 1

x2: 1 1 2 2

y: 4 5 7 5

Now we have TWO inputs! Let's try: y = 2×x1 + 1×x2 + 1

Let me try again: y = x1 + 2×x2 + 1

One more try: y = 2×x1 + x2 + 1

Actually: y = x1 + x2 + 2

Pattern: y = x1 + x2 + 2

See? With two inputs, it's harder! But the idea is the same.

Puzzle 5: Real World — Temperature vs Ice Cream

Look at the pink dots. As temperature goes up, ice cream sales go up too! This makes sense — hot days make people want cold treats.

Pattern: More heat = More ice cream sales

Puzzle 6: The Challenge

plainCopy

x: 1 2 3 4 5 6

y: 2 5 10 17 26 37

These numbers grow fast, but not as fast as Puzzle 3. Let's check differences:

The differences are 3, 5, 7, 9, 11 — odd numbers! This is a clue.

Try y = x² + 1:

Pattern: y = x² + 1

5. How a Machine "Sees" a Pattern

Machines don't have eyes or intuition. They find patterns through math. But the process is surprisingly similar to how you just solved the puzzles!

Step 1: Raw Data

The machine gets numbers. Just like you saw the puzzle tables, the machine sees (x, y) pairs. At first, it has no idea what they mean.

Step 2: Look for Relationship

The machine checks: When x goes up, does y go up? Down? Stay the same? It calculates how x and y move together.

Step 3: Guess the Pattern

The machine tries a formula. For example: "Maybe y = 2x - 1?" It draws a line and sees how close it gets to the data points.

Step 4: Calculate Errors

The machine measures how far each point is from its guess. This is called the error. Small error = good guess. Big error = bad guess.

Step 5: Predict on New Data

Once the error is small enough, the machine uses its pattern to predict y for new x values it has never seen.

Step 6: Verify

If the prediction matches reality, the pattern works! If not, go back to Step 3 and try again.

Machine learning = Guess → Check Error → Adjust → Repeat until good enough

6. Real-World Patterns Are Everywhere

Patterns aren't just math puzzles. They're all around us!

Example 1: Study Hours vs Exam Score

The more you study, the better you score. It's not perfect (some people study efficiently, some don't), but the trend is clear.

Pattern: Score ≈ 5 × Hours + 40

If you study 5 hours, expected score ≈ 5(5) + 40 = 65. Study 8 hours ≈ 80!

Example 2: Age vs Height (Children)

Kids grow as they age. The pattern is strong for children, though it slows down and stops in adulthood.

Pattern: Height ≈ 6.4 × Age + 70 cm

A 5-year-old is about 6.4(5) + 70 = 102 cm tall.

Example 3: Temperature vs Coffee Sales

Here's an interesting one! As temperature goes UP, coffee sales go DOWN. This is called an inverse relationship or negative correlation.

Pattern: Coffee ≈ -2.2 × Temp + 104

On a 30°C day, coffee sales ≈ -2.2(30) + 104 = 38 cups. On a 10°C day, ≈ 82 cups!

Example 4: Apples vs Oranges

Even with the same weight, apples and oranges have different prices! This shows that the pattern depends on WHAT you're looking at, not just the numbers.

Apple pattern: Price ≈ 0.02 × Weight

Orange pattern: Price ≈ 0.03 × Weight + 1

Oranges cost more per gram. The machine learns different patterns for different things!

7. The Four Cases: What Makes Patterns Hard to Find?

Given different situations, there are four levels of pattern-finding difficulty.

Case 1: Perfect Pattern — Easy

The data follows the rule exactly. No noise, no exceptions.

Example: y = 2x + 1 with perfect data

Machine finds it instantly. Humans too!

Case 2: Noisy Pattern — Medium

Real data has noise. The points wiggle around the line but still follow the trend.

Example: House prices, exam scores

Machine needs to find the "best fit" line through the noise

Case 3: Hidden Pattern — Hard

The pattern is not obvious. It might be exponential, or involve multiple inputs, or need transformation.

Example: y = x² + 1, or y = sin(x)

Machine (and humans!) need more tries to find the right formula

Case 4: No Pattern — Impossible

The data is truly random. No rule connects x to y.

Example: Lottery numbers, coin flips

Even the smartest machine cannot predict randomness!

Summary: The clarity of the pattern tells you how easy prediction is. Perfect patterns are easy. Noisy patterns need statistics. Hidden patterns need clever algorithms. No patterns mean prediction is impossible. This is the master key to knowing when machine learning will work.

Python Implementation: Pattern Finder from Scratch

Below is a complete, zero-library implementation that plays the "Spot the Pattern" game. It tries different formulas and picks the one with the smallest error.

```

python

# ============================================

# BLOG 3: SPOT THE PATTERN - INTUITION BUILDER

# Zero-library pattern finding

# ============================================

def check_pattern(data, w1, w2=0, b=0, pattern_type="linear"):

"""

Check if a pattern fits the data.

Returns total error.


pattern_type options:

- "linear": y = w1*x + b

- "linear_2d": y = w1*x1 + w2*x2 + b

- "squared": y = w1*(x**2) + b

- "multiplied": y = w1 * x

"""

total_error = 0


for point in data:

if len(point) == 2: # Single input: (x, y)

x, y_actual = point

if pattern_type == "linear":

y_predicted = w1 * x + b

elif pattern_type == "squared":

y_predicted = w1 * (x ** 2) + b

elif pattern_type == "multiplied":

y_predicted = w1 * x

else:

y_predicted = w1 * x + b

else: # Two inputs: (x1, x2, y)

x1, x2, y_actual = point

y_predicted = w1 * x1 + w2 * x2 + b


error = (y_predicted - y_actual) ** 2

total_error += error


return total_error

def find_best_pattern(data, pattern_type="linear"):

"""

Try many combinations of weights and find the best one.

This is brute-force learning!

"""

print(f"\n{'='*60}")

print(f"SEARCHING FOR PATTERN: {pattern_type}")

print(f"{'='*60}")


best_error = float('inf')

best_w1, best_w2, best_b = 0, 0, 0


# Try different weights (brute force)

if pattern_type == "linear_2d":

# Two inputs: try w1, w2, b

for w1 in [i * 0.5 for i in range(-5, 11)]:

for w2 in [i * 0.5 for i in range(-5, 11)]:

for b in range(-5, 11):

error = check_pattern(data, w1, w2, b, pattern_type)

if error < best_error:

best_error = error

best_w1, best_w2, best_b = w1, w2, b

else:

# One input: try w1 and b

for w1 in [i * 0.5 for i in range(-5, 11)]:

for b in range(-10, 11):

error = check_pattern(data, w1, 0, b, pattern_type)

if error < best_error:

best_error = error

best_w1, best_b = w1, b


return best_w1, best_w2, best_b, best_error

def show_results(data, w1, w2, b, pattern_type, puzzle_name):

"""Display the found pattern and verify it."""

print(f"\n{'='*60}")

print(f"PUZZLE SOLVED: {puzzle_name}")

print(f"{'='*60}")


if pattern_type == "linear_2d":

print(f"Pattern found: y = {w1}*x1 + {w2}*x2 + {b}")

elif pattern_type == "squared":

print(f"Pattern found: y = {w1}*x² + {b}")

elif pattern_type == "multiplied":

print(f"Pattern found: y = {w1}*x")

else:

print(f"Pattern found: y = {w1}*x + {b}")


print(f"\nVerification:")

print(f"{'Input':>15} {'Actual y':>10} {'Predicted':>12} {'Match?':>8}")

print("-" * 55)


for point in data:

if len(point) == 2:

x, y_actual = point

if pattern_type == "squared":

y_pred = w1 * (x ** 2) + b

elif pattern_type == "multiplied":

y_pred = w1 * x

else:

y_pred = w1 * x + b

match = "YES" if abs(y_pred - y_actual) < 0.1 else "NO"

print(f"x={x:>12} {y_actual:>10} {y_pred:>12.1f} {match:>8}")

else:

x1, x2, y_actual = point

y_pred = w1 * x1 + w2 * x2 + b

match = "YES" if abs(y_pred - y_actual) < 0.1 else "NO"

print(f"({x1},{x2}){'':>8} {y_actual:>10} {y_pred:>12.1f} {match:>8}")


return w1, w2, b

# ============================================

# SOLVE ALL THE PUZZLES!

# ============================================

print("=" * 60)

print("WELCOME TO THE PATTERN SPOTTING GAME!")

print("=" * 60)

print("Can the machine find the hidden patterns? Let's see!")

# Puzzle 1: y = x + 2

print("\n" + "🧩" * 20)

puzzle1_data = [(1, 3), (2, 4), (3, 5), (4, 6), (5, 7)]

w1, _, b, _ = find_best_pattern(puzzle1_data, "linear")

show_results(puzzle1_data, w1, 0, b, "linear", "Puzzle 1: y = x + 2")

# Puzzle 2: y = 2x

print("\n" + "🧩" * 20)

puzzle2_data = [(1, 2), (2, 4), (3, 6), (4, 8), (5, 10)]

w1, _, b, _ = find_best_pattern(puzzle2_data, "multiplied")

show_results(puzzle2_data, w1, 0, b, "multiplied", "Puzzle 2: y = 2x")

# Puzzle 3: y = x²

print("\n" + "🧩" * 20)

puzzle3_data = [(1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]

w1, _, b, _ = find_best_pattern(puzzle3_data, "squared")

show_results(puzzle3_data, w1, 0, b, "squared", "Puzzle 3: y = x²")

# Puzzle 4: y = x1 + x2 + 2

print("\n" + "🧩" * 20)

puzzle4_data = [(1, 1, 4), (2, 1, 5), (3, 2, 7), (1, 2, 5)]

w1, w2, b, _ = find_best_pattern(puzzle4_data, "linear_2d")

show_results(puzzle4_data, w1, w2, b, "linear_2d", "Puzzle 4: y = x1 + x2 + 2")

# Puzzle 5: Real world - noisy data

print("\n" + "🧩" * 20)

print("\nReal-world example: Study hours vs exam score (with noise!)")

study_data = [(1, 45), (2, 55), (3, 50), (4, 65), (5, 70), (6, 78), (7, 85)]

w1, _, b, err = find_best_pattern(study_data, "linear")

show_results(study_data, w1, 0, b, "linear", "Real World: Study Hours vs Score")

print(f"\nNote: Some errors because real data has noise!")

print(f"Total error: {err:.1f} (not zero, but small = good pattern!)")

# Summary

print("\n" + "=" * 60)

print("WHAT DID WE LEARN?")

print("=" * 60)

print("""

✅ Machines find patterns by trying many formulas

✅ They pick the one with the SMALLEST error

✅ Perfect data = zero error

✅ Real data = small error (noise is normal!)

✅ More inputs = harder puzzle, but same idea

This is the heart of machine learning:

FIND THE PATTERN → CHECK ERRORS → USE TO PREDICT

""")

```

✅ Machines find patterns by trying many formulas

✅ They pick the one with the SMALLEST error

✅ Perfect data = zero error

✅ Real data = small error (noise is normal!)

✅ More inputs = harder puzzle, but same idea

This is the heart of machine learning:

FIND THE PATTERN → CHECK ERRORS → USE TO PREDICT

The Four Cases: Pattern Difficulty

Case

Pattern Type

Error

Example

Can Machine Find It?

1

Perfect linear

Zero

y = 2x + 1

Yes — instantly

2

Noisy linear

Small

House prices

Yes — best fit line

3

Non-linear

Depends

y = x² + 1

Yes — if we try right formula

4

No pattern

Huge

Random data

No — impossible!

Complete solution = look at data + guess a pattern + measure error + pick the best guess. The pattern complexity comes from the data. The accuracy comes from checking errors carefully.

Why This Builds Intuition

It may seem like child's play, but this "spot the pattern" game is exactly what machines do — just with millions of numbers instead of five. When you understand that:

then you understand the heart of machine learning. Everything else — neurons, weights, gradients — is just machinery for doing this faster and better.

The duckling that imprints on Lorenz, the student who notices that studying more leads to better grades, the machine that learns house prices from square footage — all are doing the same thing: finding patterns in data and using them to predict the future.

Try It Yourself!

Exercise 1: Find the pattern:

x: 2 4 6 8 10

y: 7 11 15 19 23

Exercise 2: Find the pattern with TWO inputs:

x1: 1 2 3 2

x2: 2 1 3 2

y: 5 4 7 6

Exercise 3: Is there a pattern here? Or is it random?

x: 1 2 3 4 5

y: 3 1 4 1 5

(Hint: Think about the digits of pi!)

References & Further Reading

Anil Ananthaswamy. Why Machines Learn: The Elegant Math Behind Modern AI. Dutton, 2024.

Konrad Lorenz. "The Companion in the Bird's World." Auk, 1935. (The original imprinting study that inspired the chapter's opening)

3Blue1Brown. Neural Networks series, Chapter 1: But what is a neural network? YouTube.

Khan Academy. Patterns and relationships in data. khanacademy.org/math

More Machine-Learning tutorials

All tutorials · Try the free PySpark compiler · Practice challenges