Prerequisite
Before diving in, you should be comfortable with the Pythagorean theorem: in a right-angled triangle, the square of the hypotenuse (the longest side) equals the sum of the squares of the other two sides — written as a² + b² = c². We will use this to calculate the length (magnitude) of a vector.
1.1 Introduction
Linear algebra is a branch of mathematics that studies linear equations and how they can be represented using vectors and matrices. It may sound abstract at first, but the ideas are deeply intuitive once you see them through the right examples.
At its heart, linear algebra is about two things: direction and magnitude. To understand why we even need it, let's start with a simple distinction — the difference between a scalar and a vector.
Scalars vs. Vectors
Imagine a man walks five miles. That single number — 5 — is called a scalar. It tells us how much, but nothing about which way. Scalars are just ordinary numbers: temperature, mass, time, speed.
Now imagine we are told that the man walked five miles in a north-easterly direction. Suddenly we have two pieces of information: the distance (magnitude) and the direction. This combined quantity is a vector — and that is the fundamental building block of linear algebra.
Think of it this way: a scalar answers "how much?" while a vector answers "how much AND which way?"
Representing a Vector
We can represent the man's walk on a coordinate plane. His journey from the starting point (0, 0) to the destination (4, 3) is drawn as an arrow. This arrow has:
- An x-component of 4 (4 miles east)
- A y-component of 3 (3 miles north)
Figure 1.1 — A vector from (0,0) to (4,3) representing a 5-mile north-east walk.
Notice that the vector, the x-axis, and the y-axis together form a right-angled triangle. Using the Pythagorean theorem, the length (magnitude) of the vector is:
|v| = √(4² + 3²) = √(16 + 9) = √25 = 5 miles
The magnitude of a vector is simply the straight-line distance from its starting point to its end point — always calculated using the Pythagorean theorem.
Vector Addition
After reaching (4, 3), the man turns and continues walking — this time arriving at (6, 9). This second leg of the journey is another vector, with an x-component of 2 and a y-component of 6.
How do we combine the two walks into one total journey? We use vector addition. The rule is beautifully simple: add the x-components together and add the y-components together.
(4, 3) + (2, 6) = (6, 9)
Visually, vector addition is like placing the tail of the second arrow at the tip of the first arrow — and the result (called the resultant vector) goes from the very beginning to the very end.
Figure 1.2 — Vector addition: the combined walk from (0,0) to (4,3) and then to (6,9).
1.2 Vector Space
Now that you understand what a vector is, let's zoom out and ask: where do vectors live? The answer is a vector space — and understanding this concept unlocks almost all of linear algebra.
A vector space is like a structured "world" or "environment" in which vectors exist. This world comes with a clear set of rules that govern how vectors can be moved, added, and scaled — while always preserving their meaning (their length and direction).
A great way to picture it: the vector is the arrow, but the vector space is the entire plane (or 3D space, or n-dimensional space) in which that arrow exists.
Why Does the Vector Space Matter?
Going back to our walking example: the man's walk is represented by the vector (0, 0) → (4, 3). But this vector doesn't float in a vacuum — it exists inside the 2-dimensional coordinate plane, and that plane is the vector space.
The coordinate plane provides three essential ingredients that give the vector its meaning:
- Directions: east–west (x-axis) and north–south (y-axis).
- Distance: a consistent way to measure how far apart points are.
- Rules for combination: how to add and scale vectors in a meaningful way.
Without the surrounding plane, statements like "4 miles east" and "3 miles north" would have no frame of reference and no meaning. The vector space is what makes it all coherent.
Vectors Can Be Moved Freely
One of the most important properties of a vector space is that a vector is defined only by its direction and magnitude — not by where it starts. This means we can slide the arrow anywhere in the plane without changing the vector.
For example, these two arrows represent exactly the same vector:
(0, 0) → (4, 3) is the same vector as (2, 1) → (6, 4)
Both have the same direction and length. The freedom to translate vectors is not a property of the arrow itself — it is a property of the vector space it lives in.
Closure: Staying Inside the Space
When the man walks the first leg (4, 3) and then the second leg (2, 6), the combined result (6, 9) is still a valid point in the same plane. This is called closure — adding two vectors always produces another vector inside the same vector space.
Closure is one of the defining rules of a vector space: you can never "escape" the space by adding or scaling vectors.
Scaling: Making Vectors Bigger or Smaller
Scaling a vector means multiplying it by a number (called a scalar). If the man repeated his walk twice, the resulting displacement would be:
2 × (4, 3) = (8, 6)
The direction is the same, but the magnitude doubles. Scaling always keeps the result inside the same vector space — another example of closure.
1.3 Vector Representation
A vector can be written down in three different ways depending on what you want to emphasize — geometry, computation, or structure. All three describe the same mathematical object, just from different angles.
1. Geometric Representation — v = (x, y)
In this form, a vector is written as an ordered pair, for example v = (4, 3). It means: start at the origin and end at the point (4, 3). This is the most visual form — it directly shows you the arrow in space.
This representation is most useful when you want to draw the vector, think about angles, or understand direction and magnitude intuitively. It is the form we have been using throughout this chapter.
2. Array Representation — v = [x, y]
Here, the same vector is written as a column or row array, such as v = [4, 3]. The vector is now treated as an ordered list of numbers rather than a geometric arrow.
This form is built for computation. When you write code to process vectors in machine learning, numerical methods, or data science, this is the representation you will use. It is efficient, easy to store, and easy to transform using matrix operations.
3. Basis Vector Representation — v = x·î + y·ĵ
In this form, a vector is expressed as a combination of two special unit vectors: î (pointing one unit in the x-direction) and ĵ (pointing one unit in the y-direction). Our example becomes:
v = 4î + 3ĵ
This says: the vector is made up of 4 units of î (east) and 3 units of ĵ (north). These î and ĵ are called basis vectors — the fundamental building blocks from which all other vectors in the space are constructed.
This representation becomes especially important in advanced linear algebra when you change the coordinate system (change of basis) or work in higher-dimensional spaces.
Remember: all three representations describe the same vector. Choosing which form to use depends entirely on what you are trying to do — visualize, compute, or reason theoretically.
Quick Summary
Here is a concise recap of the key ideas from this chapter:
- Scalar: a single number with magnitude only (e.g., 5 miles).
- Vector: a quantity with both magnitude and direction (e.g., 5 miles north-east).
- Magnitude: the length of a vector, calculated using the Pythagorean theorem.
- Vector Addition: add x-components and y-components separately.
- Vector Space: the structured plane (or higher-dimensional space) in which vectors live, with rules for addition and scaling.
- Closure: adding or scaling vectors always stays within the same vector space.
- Basis Vectors (î, ĵ): the fundamental unit-length building blocks of a vector space.
References
Ananthaswamy, Anil. Why Machines Learn. Dutton, 2018.
Wikipedia — Linear Algebra. https://en.wikipedia.org/wiki/Linear_algebra
End of Dot Product Blog — Linear Algebra Series