Vectors& Linear Equations

Linear-Algebra tutorial · PySpark.in

1. What Is a Vector?

A vector is an ordered list of numbers. That is the algebraic definition, and it is all we need to get started. You may have encountered vectors in a physics class as arrows with direction and magnitude, and that geometric picture is valid and useful — but in linear algebra, we treat a vector simply as a column of numbers that we can add, scale, and combine.

v = [ 3 ]

[ 1 ]

This vector v has two components: 3 and 1. You can picture it as an arrow from the origin to the point (3, 1) in the plane. But you can just as well think of it as a pair of numbers obeying certain rules of arithmetic. Both pictures are valuable, and switching between them fluently is a key skill in linear algebra.

2. Two Fundamental Operations

Everything in linear algebra — from solving equations to understanding transformations — is built on just two basic operations with vectors. Master these two operations and everything else follows.

Scalar Multiplication

Multiply every component of the vector by the same number, called a scalar. If we scale the column vectors from our system — say v₁ = (1, 3) — by the scalar 3:

3 · [ 1 ] = [ 3 ]

[ 3 ] [ 9 ]

Geometrically, this stretches the arrow to three times its original length, pointing in exactly the same direction. Scaling by a negative number reverses the direction. Scaling by zero collapses the vector to the zero vector at the origin.

Vector Addition

Add two vectors by adding their components one by one. Given the two column vectors from our example system, v₁ = (1, 3) and v₂ = (-2, 2):

3 · [ 1 ] + 1 · [-2 ] = [ 3 ] + [-2 ] = [ 1 ]

[ 3 ] [ 2 ] [ 9 ] [ 2 ] [ 11 ]

Geometrically, vector addition follows the parallelogram rule: place the tail of the second vector at the tip of the first, and the result is the diagonal of the parallelogram they form. The combination 3v₁ + 1v₂ reaches precisely the target vector b = (1, 11).

These two operations — scaling and adding — combine to form linear combinations. A linear combination of vectors is any expression c₁v₁ + c₂v₂ + ... + cₙvₙ, where the c's are scalars. This single idea is the foundation of the entire subject.

3. The Column Picture of Ax = b

Recall the two-equation system from Blog 2. In the column picture, we rewrote it as a linear combination of column vectors:

x · [ 1] + y · [-2] = [ 1]

[ 3] [ 2] [ 11]

This is a linear combination question: what values of x and y combine the two column vectors to produce the right-hand side b = (1, 11)? The answer x = 3, y = 1 says: take 3 copies of column 1 and 1 copy of column 2. Their sum lands precisely on b.

The column picture is powerful because it reframes equation-solving as vector-reaching. The question is no longer just about satisfying equations — it is about whether the target vector b can be reached by combining the columns of A in some way.

The column picture of Ax = b: a linear combination of the columns of A produces the vector b. This reframes every linear system as a question about linear combinations — can we reach b by combining the columns of A?

4. The Row Picture of Ax = b

In the row picture, each equation of the system describes a geometric object. In 2D, each linear equation draws a line. In 3D, it draws a plane. The dot product gives the equation of each object precisely:

(row 1) · x = b₁ means 1·x - 2·y = 1

(row 2) · x = b₂ means 3·x + 2·y = 11

Each row of A dots with the unknown vector x and must equal the corresponding entry of b. When b = 0, all these planes pass through the origin. When b is nonzero, each plane is shifted away from the origin by a specific amount determined by its right-hand side value.

The row picture of Ax = b: m equations from m rows give m planes meeting at x. A dot product gives the equation of each plane: (row i) · x = bᵢ. The solution is the point where all those planes intersect.

5. The Coefficient Matrix: Same Numbers, Two Pictures

The brilliance of writing Ax = b is that it contains both pictures simultaneously. The same matrix A, looked at row by row, gives you the row picture of intersecting planes. Looked at column by column, it gives you the column picture of combining vectors. Same numbers, two illuminating perspectives — this is what Strang means when he says linear algebra offers multiple ways to see the same truth.

Coefficient matrix: A = [ 1 -2 ]

[ 3 2 ]

Row view: each row of A · x = bᵢ draws a line

Column view: x·col1 + y·col2 = b is a linear combination

6. The Column Space: Which Vectors b Are Reachable?

A natural question arises once you understand the column picture: for which vectors b can we actually solve Ax = b? The answer is all vectors b that are linear combinations of the columns of A. The collection of all such reachable vectors forms the column space of A.

If A has two columns in 2D that point in genuinely different directions (not parallel to each other), then every vector in 2D is reachable — the column space is the whole plane. But if the two columns point in the same direction, only vectors along that single line are reachable. No matter what coefficients you choose, you cannot escape that line.

Geometric intuition: independent vectors are like roads going in different directions — together they can take you anywhere in the plane. Dependent vectors all point the same way — no matter how far you travel, you stay on one road.

7. Linear Independence and Dependence

Two column vectors are independent when neither is a scalar multiple of the other — they point in genuinely different directions. Three column vectors u, v, w are independent when no one of them lies in the span of the other two. Independence means the columns collectively point in truly distinct directions and together can reach any point in their ambient space.

Dependence means at least one vector is already a combination of the others. If w* = -u - v, then adding w* to the set brings no new direction. The three vectors u, v, w* all lie in the same plane, and their combinations can only fill that plane — they can never reach anything outside it.

u, v, w are INDEPENDENT: Ax = 0 has one solution. A is invertible.

u, v, w* are DEPENDENT: Cx = 0 has many solutions. C is singular.

This distinction — invertible versus singular, independent versus dependent — is the central theme of the entire subject. Every major theorem in linear algebra is, in some sense, a precise statement about when matrices are invertible and when they are not.

The key question for three vectors: is the third vector in the plane of the first two? If yes, they are dependent and the matrix is singular. If no, they are independent and the matrix is invertible. The dimension of the space they can reach is called the rank of the matrix.

References & Further Reading

Gilbert Strang. Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, 2016. (Chapter 2.1; Section 1.3)

3Blue1Brown. Essence of Linear Algebra, Chapters 2 & 3: Linear Combinations, Span, and Basis Vectors. YouTube, 2016.

Khan Academy. Vectors and spaces unit. khanacademy.org/math/linear-algebra

End of Blog 3 — Linear Algebra Series