Matrices

Linear-Algebra tutorial · PySpark.in

1. What Even Is a Matrix?

If you have ever stored numbers in a table — a spreadsheet, a grade sheet, a timetable — you already understand the soul of a matrix. A matrix is simply a rectangular arrangement of numbers, organised into rows (going left to right) and columns (going top to bottom), enclosed in square brackets. Nothing mysterious; just numbers in a very deliberate arrangement.

Here is one of the first examples Gilbert Strang introduces in his Introduction to Linear Algebra:

A = [ 1 2 ]

[ 3 4 ]

[ 5 6 ]

This matrix A has 3 rows and 2 columns, so we call it a 3×2 matrix. The convention is always rows first, then columns — just like reading a coordinate in (row, column) order. R before C, just like the words row and column themselves.

Think of a matrix like a classroom seating chart. The rows are the horizontal rows of desks; the columns are the vertical columns of desks. Every seat holds exactly one number, identified by its row and column address.

2. Why Do Matrices Matter?

Matrices are the workhorses of linear algebra. They do not just store data — they transform it. When you multiply a matrix by a vector, you are applying a transformation: rotating, stretching, shearing, or projecting that vector into a new position. This idea powers computer graphics, machine learning, physics simulations, and economics.

Strang puts it clearly: the matrix A acts on the vector x. The output Ax is a combination of the columns of A. That single idea — matrix times vector equals a combination of columns — is the beating heart of everything in linear algebra.

3. Reading a Matrix: Rows, Columns, and Entries

Each individual number inside a matrix is called an entry or element. We identify an entry by its position: a₁₂ means row 1, column 2. In the matrix A above, a₁₂ = 2 and a₃₁ = 5.

Rows are horizontal slices. Columns are vertical slices. When we later multiply Ax, each row of A produces one output number through a dot product with x. Each column of A is a vector that we will combine using the entries of x as weights.

Key Idea: A matrix stores both data and instructions. Its rows tell you how to compute each output via dot products; its columns tell you which directions the transformation can reach.

4. Matrix Times Vector: Two Ways to See It

Consider the 3×3 matrix A and a vector x with three components. There are two equivalent ways to compute the product Ax, and understanding both is crucial for building deep intuition.

A = [ 1 0 0 ] x = [ x1 ]

[-1 1 0 ] [ x2 ]

[ 0 -1 1 ] [ x3 ]

Row View: Dot Products

In the row view, each entry of the output vector Ax is the dot product of one row of A with the entire vector x. The first output entry equals (row 1) · x = 1·x₁ + 0·x₂ + 0·x₃ = x₁. The second equals (row 2) · x = -x₁ + x₂. The third gives x₂ subtracted from x₃. This is the method most people first learn — row by row, dot product by dot product.

Column View: Linear Combination

In the column view — the one Strang strongly prefers — Ax is a linear combination of the columns of A, where the weights are the entries of x:

Ax = x1·(column 1) + x2·(column 2) + x3·(column 3)

This rewrite is not just clever notation. Thinking column-by-column connects matrix multiplication to span, linear independence, and ultimately to solving equations. As Strang writes, linear combinations are the key to linear algebra, and Ax is always a linear combination of the columns of A.

With numbers, you can multiply Ax row by row. With ideas, columns are the good way. The column view reveals what the matrix can actually produce — which vectors it can reach, and which it cannot.

5. The Difference Matrix: A Concrete Worked Example

Strang introduces the difference matrix A as one of the most instructive examples in the whole subject. When you multiply A by x = (x₁, x₂, x₃), the output b = Ax contains the successive differences between consecutive components:

b = [ x1 ] (just x1 itself)

[ x2 - x1 ] (second minus first)

[ x3 - x2 ] (third minus second)

If x = (1, 4, 9) — the perfect squares — then b = (1, 3, 5) — the odd numbers. The matrix is literally computing the rate of change between consecutive values, just as a derivative does in calculus. The analogy runs deep: the inverse operation, summing differences, is like integration. Strang notes this connection explicitly — sums of differences are like integrals of derivatives.

The Fundamental Theorem of Calculus says integration is the inverse of differentiation. In matrix language: Ax = b and x = A⁻¹b. The difference matrix plays the role of the derivative; its inverse, the sum matrix, plays the role of the integral.

6. Invertible vs Singular Matrices

Not every matrix can be undone. A matrix A is called invertible (or non-singular) if there exists another matrix A⁻¹ such that A⁻¹A = I, where I is the identity matrix. This means that for every output vector b, there is exactly one input vector x that produces it — the transformation has no information loss.

The difference matrix A above is invertible. From any set of differences b, you can reconstruct x exactly using cumulative sums. The inverse A⁻¹ is the sum matrix.

But consider the cyclic difference matrix C, where the differences wrap around and the last entry involves x₁ - x₃. Its columns all add to zero — they are linearly dependent. The equation Cx = 0 has infinitely many solutions (any constant vector works). C has no inverse. It is singular.

Independent columns: Ax = 0 has one solution. A is invertible.

Dependent columns: Cx = 0 has many solutions. C is singular.

Think of an invertible matrix as a perfectly encoded message: every output has a unique input, and you can always decode it. A singular matrix is like a lossy compression — information is permanently destroyed and the original cannot be recovered.

7. Independence, Dependence, and What They Mean Geometrically

The three columns of the invertible matrix A span all of three-dimensional space — any vector b can be produced as a combination of those columns. The three columns of the singular matrix C all lie in a single plane; together they can only reach vectors b whose components sum to zero.

This is the deep connection between algebra and geometry. The rank of a matrix — the number of truly independent columns — tells you the dimension of the space that matrix can reach. A full-rank 3×3 matrix reaches all of 3D space. A rank-2 matrix can only reach a plane. A rank-1 matrix, only a line.

Understanding this is the gateway to Chapter 3 and beyond. Every major theorem in linear algebra — about solutions, about eigenvalues, about decompositions — traces back to this fundamental question: are the columns independent or dependent?

References & Further Reading

Gilbert Strang. Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, 2016. (Section 1.3: Matrices)

3Blue1Brown. Essence of Linear Algebra, Chapter 3: Linear Transformations and Matrices. YouTube, 2016.

Khan Academy. Matrices unit. khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices

End of Blog 1 — Linear Algebra Series