Matrix Multiplication

Linear-Algebra tutorial · PySpark.in

Matrix Multiplication

Prerequisite: Introduction to Linear Algebra | Understanding Vector Spaces

1. What Is a Matrix?

Before we multiply matrices, we need to be completely clear on what a matrix is. A matrix is simply a rectangular grid of numbers, arranged in rows and columns, surrounded by square brackets. That is it. Nothing magical — just numbers in a table.

Here is an example of a matrix:

A = [ 1 2 3 ]

[ 4 5 6 ]

This matrix has 2 rows and 3 columns, so we call it a "2 by 3" matrix, written as 2×3. The first number always tells you the rows, the second tells you the columns. Easy to remember: R before C, just like "Row, Column" in that order.

Think of a matrix like a spreadsheet. Rows go left-to-right (horizontal). Columns go top-to-bottom (vertical). A 3×4 matrix is like a spreadsheet with 3 rows and 4 columns.

Naming the Entries

Each individual number inside a matrix is called an entry or element. We refer to an entry by its position: a₁₂ means the entry in row 1, column 2. In the matrix A above, a₁₂ = 2 (row 1, column 2), and a₂₁ = 4 (row 2, column 1).

This labeling system becomes very important when we multiply matrices, because multiplication is all about picking specific rows and columns and combining them.

2. Why Do We Even Multiply Matrices?

This is the most important question a beginner can ask — and most textbooks skip the answer entirely. Let's fix that.

Imagine you run a small shop. You sell 3 products: apples, bananas, and cherries. Each week, you have a sales matrix that records how many of each product you sold on each day:

Sales = [ 10 20 15 ] (Monday)

[ 5 30 10 ] (Tuesday)

You also have a price matrix that records the price of each product (in dollars):

Price = [ 1.00 ] (apple)

[ 0.50 ] (banana)

[ 2.00 ] (cherry)

To find your total revenue for each day, you multiply units sold by price and add them up. For Monday: (10 × 1.00) + (20 × 0.50) + (15 × 2.00) = 10 + 10 + 30 = $50.

That process — multiply matching entries and add them up — is exactly what matrix multiplication does, for every row and every column, all at once. Matrix multiplication is the mathematical tool for combining multiple such calculations efficiently.

The big picture: Matrix multiplication is how we apply transformations, combine data relationships, and chain together multi-step calculations — all in one compact operation. It powers graphics (rotating 3D objects), machine learning (neural networks), economics, and physics.

3. The Dot Product — The Key Ingredient

Every single entry in a matrix multiplication result comes from a calculation called the dot product. Once you fully understand the dot product, matrix multiplication becomes mechanical and obvious. So let's nail this first.

What Is a Dot Product?

The dot product takes two lists of numbers of the same length, multiplies each pair of matching numbers together, then adds all those products up into a single number.

Take these two lists, called vectors:

a = [ 1, 2, 3 ]

b = [ 4, 5, 6 ]

Their dot product is calculated like this:

a · b = (1×4) + (2×5) + (3×6)

= 4 + 10 + 18

= 32

We matched position 1 with position 1 (1 and 4), position 2 with position 2 (2 and 5), position 3 with position 3 (3 and 6), multiplied each pair, then added. One dot product always gives back one single number.

Connect to the shop example: Monday's sales [10, 20, 15] dotted with prices [1.00, 0.50, 2.00] gives (10×1)+(20×0.5)+(15×2) = 50. That $50 is a dot product. Matrix multiplication is just doing this for every row of one matrix and every column of the other.

The Dot Product Needs Matching Lengths

You can only take the dot product of two lists if they have the same number of elements. You cannot dot [1, 2, 3] with [4, 5] because they do not match up. This constraint is the exact same rule that determines when matrix multiplication is even possible — as we'll see next.

4. The Golden Rule — When Can You Multiply Two Matrices?

Not every pair of matrices can be multiplied. There is one strict rule you must check before you do anything else. Here it is:

The Rule: To multiply matrix A (of size m×n) by matrix B (of size n×p), the number of COLUMNS in A must equal the number of ROWS in B. If they match, the result C = A×B will have size m×p.

In other words: if A is m×n and B is n×p, then C = AB is m×p. The inner two dimensions (both n) must match, and they "disappear" in the result. Only the outer dimensions (m and p) survive.

Let's look at some examples:

(2×3) × (3×4) → Valid! Result is (2×4) ← inner both 3, match ✓

(2×3) × (4×3) → Invalid! Inner 3 ≠ 4 ← inner 3 and 4, no match ✗

(4×2) × (2×5) → Valid! Result is (4×5) ← inner both 2, match ✓

(1×3) × (3×1) → Valid! Result is (1×1) ← inner both 3, match ✓

Memory trick: write the dimensions side by side — (m × n)(n × p). The two middle numbers must be the same. Cross them out. What remains, m × p, is the size of your answer.

Watch out: A×B and B×A are almost never equal — and sometimes only one of them even exists! Matrix multiplication is NOT commutative. Order always matters.

5. How to Multiply Two Matrices — Step by Step

Now for the core skill. Here is the procedure, stated once clearly: each entry C[i][j] in the result matrix is the dot product of row i of A with column j of B. That is the entire rule. Let's build this up slowly with a concrete example.

Our Working Example

We will multiply these two matrices:

A = [ 1 2 ] B = [ 5 6 ]

[ 3 4 ] [ 7 8 ]

A is 2×2 and B is 2×2. The inner dimensions both equal 2, so multiplication is valid. The result C = A×B will also be 2×2. That means C has 4 entries: C[1][1], C[1][2], C[2][1], and C[2][2]. We will calculate each one.

Finding C[1][1] — Row 1 of A · Column 1 of B

To find the entry in row 1, column 1 of the result, we take row 1 of A and column 1 of B and compute their dot product.

Row 1 of A = [ 1 2 ]

Column 1 of B = [ 5 7 ] (reading down column 1)

C[1][1] = (1×5) + (2×7) = 5 + 14 = 19

Finding C[1][2] — Row 1 of A · Column 2 of B

Same row 1 of A, but now we pair it with column 2 of B:

Row 1 of A = [ 1 2 ]

Column 2 of B = [ 6 8 ] (reading down column 2)

C[1][2] = (1×6) + (2×8) = 6 + 16 = 22

Finding C[2][1] — Row 2 of A · Column 1 of B

Row 2 of A = [ 3 4 ]

Column 1 of B = [ 5 7 ]

C[2][1] = (3×5) + (4×7) = 15 + 28 = 43

Finding C[2][2] — Row 2 of A · Column 2 of B

Row 2 of A = [ 3 4 ]

Column 2 of B = [ 6 8 ]

C[2][2] = (3×6) + (4×8) = 18 + 32 = 50

Assembling the Result

We now place all four entries into the result matrix C:

C = A × B = [ 19 22 ]

[ 43 50 ]

The pattern to remember: "Row i of A, dotted with Column j of B, gives the entry at row i, column j of C." Repeat this for every (i, j) position in the result.

6. A Bigger Example (2×3 times 3×2)

Let's try a slightly larger example to make sure the process is solid. This time A and B have different shapes.

A = [ 1 2 3 ] (2 rows, 3 columns)

[ 4 5 6 ]

B = [ 7 8 ] (3 rows, 2 columns)

[ 9 10 ]

[11 12 ]

A is 2×3 and B is 3×2. The inner dimensions are both 3 — they match! So multiplication is valid. The result C will be 2×2 (the outer dimensions: 2 rows and 2 columns).

Now we calculate each of the 4 entries in C:

C[1][1]: Row 1 of A · Col 1 of B = (1×7)+(2×9)+(3×11) = 7+18+33 = 58

C[1][2]: Row 1 of A · Col 2 of B = (1×8)+(2×10)+(3×12) = 8+20+36 = 64

C[2][1]: Row 2 of A · Col 1 of B = (4×7)+(5×9)+(6×11) = 28+45+66 = 139

C[2][2]: Row 2 of A · Col 2 of B = (4×8)+(5×10)+(6×12) = 32+50+72 = 154

Final result:

C = A × B = [ 58 64 ]

[ 139 154 ]

Notice how we always need 3 multiplications and 2 additions for each entry, because the matching inner dimension was 3. In general, each entry requires n multiplications and (n−1) additions, where n is the inner dimension.

7. One Matrix, Two Steps — Chaining Linear Systems

This section reveals the deepest reason why matrix multiplication is defined the way it is. Everything we have practised so far has been mechanics. Now we get to the meaning. The core insight is this: when you multiply two matrices together, the result is a brand-new single matrix that does the work of both originals, back to back, in one shot. To see why, we first need to understand what one matrix "does" on its own.

What Is a Linear System in Matrix Form?

A linear system is just a set of equations where every variable appears only by itself — no squares, no products of variables, no weird functions. Here is a simple example with two equations and two unknowns x₁ and x₂:

y₁ = 2·x₁ + 1·x₂

y₂ = 1·x₁ + 3·x₂

This system takes two inputs (x₁ and x₂) and produces two outputs (y₁ and y₂). Notice that every output is just a weighted sum of the inputs — that is what makes it linear. We can package this system into a clean matrix equation:

y = A · x where A = [ 2 1 ] and x = [ x₁ ] and y = [ y₁ ]

[ 1 3 ] [ x₂ ] [ y₂ ]

Writing it this way — y = Ax — means: "take the input vector x, pass it through the transformation described by matrix A, and receive the output vector y." The matrix A completely encodes the linear system. Every row of A holds the coefficients for one output equation.

Concrete check: if x = [1, 2], then y = Ax gives y₁ = (2×1)+(1×2) = 4 and y₂ = (1×1)+(3×2) = 7. The matrix did both equations simultaneously — that is why the matrix form is so compact and powerful.

Now Add a Second System — and Chain Them

Suppose the outputs y₁ and y₂ from our first system are not the final answer. They are fed as inputs into a second linear system, which produces final outputs z₁ and z₂:

z₁ = 1·y₁ + 1·y₂

z₂ = 2·y₁ + (−1)·y₂

In matrix form, this second system is:

z = B · y where B = [ 1 1 ]

[ 2 -1 ]

So our full pipeline is: start with x → apply A to get y → apply B to get z. Written in equations that is:

z = B · y = B · (A · x) = (B · A) · x

That last step is the key. Because matrix multiplication is associative (the grouping can be rearranged — more on this in the next section), we can compute B·A first and get a single new matrix C = BA. Then the entire two-step pipeline collapses to just:

z = C · x where C = B · A

The single matrix C permanently bakes both steps in. You never need to visit y at all — C takes x directly to z in one operation.

Working It Out — The Full Calculation

Let us compute C = BA with our actual numbers and then verify it does the same thing as the two-step process.

C = B · A = [ 1 1 ] × [ 2 1 ]

[ 2 -1 ] [ 1 3 ]

C[1][1] = (1×2) + (1×1) = 3

C[1][2] = (1×1) + (1×3) = 4

C[2][1] = (2×2) + (−1×1) = 3

C[2][2] = (2×1) + (−1×3) = −1

C = [ 3 4 ]

[ 3 -1 ]

Now let us verify with a specific input. Let x = [1, 2].

The slow way — going through y first:

Step 1: y = A·x = [ 2 1 ]·[ 1 ] = [ 4 ]

[ 1 3 ] [ 2 ] [ 7 ]

Step 2: z = B·y = [ 1 1 ]·[ 4 ] = [ 11 ]

[ 2 -1 ] [ 7 ] [ 1 ]

The fast way — using the combined matrix C directly:

z = C·x = [ 3 4 ]·[ 1 ] = [ (3×1)+(4×2) ] = [ 11 ]

[ 3 -1 ] [ 2 ] [ (3×1)+(−1×2) ] [ 1 ]

Both routes give exactly z = [11, 1]. The combined matrix C = BA is not just an approximation or a shortcut — it is exactly equivalent to running both systems one after the other. We have proven this with a concrete example.

The profound insight: matrix multiplication is not just an arithmetic operation. It is the mathematical language for composing two linear transformations into one. The result matrix does not just "relate" to the two original matrices — it is their combined effect, encoded as a single object. This is exactly why the operation is defined using dot products of rows and columns and not some simpler rule.

Why the Order B·A and Not A·B?

You might wonder: why is the combined matrix C = B·A (B on the left, A on the right) rather than A·B? Think about the pipeline: we apply A first and B second. When we write z = B·(A·x), A sits closest to the input x — so it acts first. B sits outermost — so it acts second. In the combined matrix B·A, A is still on the right (acts first) and B is on the left (acts second). The order in matrix notation reads right to left, which is the reverse of how we describe the steps in plain English.

Order trap: "First apply A, then apply B" becomes the combined matrix B·A — not A·B. The transformation applied second sits on the left. This is one of the most common sources of errors when building transformation chains in graphics, robotics, and machine learning.

A Real-World Example — Manufacturing Pipeline

Imagine a factory where raw materials [x₁, x₂] go through two processing stages. Stage 1 (matrix A) converts raw materials into intermediate parts. Stage 2 (matrix B) assembles intermediate parts into finished products. Normally, you would compute the intermediate parts first, then assemble them. But if you run this same factory every single day with thousands of different input combinations, you would quickly compute the combined matrix C = BA once and store it. From then on, every new batch of raw materials can go directly to finished products in a single matrix multiplication — no intermediate step needed. That is a massive saving when the pipeline runs millions of times.

This is exactly what happens in a neural network: multiple layers, each described by its own weight matrix, are composed together during training into combined representations. It is what happens in 3D graphics where rotation, scaling, and translation matrices are pre-multiplied into one world-transform matrix applied to every vertex. The idea is always the same — chain the systems first, then apply the result.

8. Properties of Matrix Multiplication

Matrix multiplication follows some familiar rules and breaks some others you might expect from normal arithmetic. Let's go through each one carefully.

Property 1: It Is NOT Commutative (AB ≠ BA)

With regular numbers, 3 × 5 = 5 × 3. Order does not matter. With matrices, order matters enormously. In fact, AB and BA can give completely different results — and sometimes only one of them even exists.

Quick demonstration. Take:

A = [ 1 2 ] B = [ 0 1 ]

[ 0 1 ] [ 1 0 ]

AB = [ 2 1 ] but BA = [ 0 1 ]

[ 1 0 ] [ 1 2 ]

AB ≠ BA. Always keep track of which matrix is on the left and which is on the right.

Common beginner mistake: writing A×B when you actually need B×A (or vice versa). In applications like transformations, the order defines the meaning completely.

Property 2: It IS Associative ((AB)C = A(BC))

Even though order matters, you can freely regroup when multiplying three or more matrices. (AB)C gives the same result as A(BC). This means if you have a chain of matrix multiplications, you can choose whichever grouping is most computationally convenient.

Property 3: It IS Distributive (A(B+C) = AB + AC)

Matrix multiplication distributes over addition, just like regular numbers: A(B + C) = AB + AC and (A + B)C = AC + BC. This property is used frequently in proofs and simplifications.

Property 4: The Identity Matrix

There is a special matrix called the identity matrix, denoted I, which works like the number 1 in regular multiplication: A × I = I × A = A. It has 1s on the main diagonal (top-left to bottom-right) and 0s everywhere else. Here is the 2×2 identity:

I = [ 1 0 ]

[ 0 1 ]

Multiplying any matrix by I leaves it completely unchanged. The identity matrix plays an important role when we study matrix inverses.

Property 5: The Zero Matrix

Just like multiplying any number by 0 gives 0, multiplying any matrix by the zero matrix (all entries are 0) gives a zero matrix. However — and this is a critical difference from regular numbers — it is possible for A×B = 0 even if neither A nor B is the zero matrix. This is one of those ways matrices genuinely behave differently from numbers.

9. Special and Important Cases

Case 1: Multiplying a Matrix by a Vector

You will encounter this constantly in linear algebra. A vector is just a matrix with a single column. Multiplying a 2×2 matrix by a 2×1 vector gives a 2×1 vector:

[ 1 2 ] × [ x ] = [ 1x + 2y ]

[ 3 4 ] [ y ] [ 3x + 4y ]

This is how linear transformations work — the matrix transforms the input vector into a new output vector. When you rotate a 3D object on screen, a 3×3 matrix is being multiplied by thousands of position vectors, once per frame.

Case 2: Row Vector times a Matrix (1×n times n×p)

A row vector — a matrix with a single row — can multiply a matrix from the left, giving another row vector. This appears often in statistics and machine learning, where data rows are multiplied by weight matrices.

[ a b ] × [ 1 2 ] = [ a + 3b 2a + 4b ]

[ 3 4 ]

Case 3: Squaring a Matrix (A²)

A² simply means A×A. As long as A is a square matrix (same number of rows and columns), this is always valid. A² represents applying transformation A twice in a row — useful in physics, graph theory, and many other fields.

Case 4: Multiplying a 1×n by n×1 (Getting a Single Number)

This special case gives a 1×1 result — effectively just a single number. You will recognise it immediately as the dot product. This is the link: the dot product of two vectors is the same thing as multiplying a row vector by a column vector.

[ 1 2 3 ] × [ 4 ] = (1×4) + (2×5) + (3×6) = 32

[ 5 ]

[ 6 ]

10. The 5 Most Common Beginner Mistakes

These mistakes appear again and again. Being aware of them saves a lot of time and frustration.

Mistake 1: Multiplying Entry by Entry (Element-wise)

Beginners sometimes think matrix multiplication means multiplying the entry at position (1,1) of A with the entry at position (1,1) of B, and so on. That operation does exist — it's called the Hadamard product — but it is not standard matrix multiplication. Standard multiplication uses dot products of rows and columns, not individual matching entries.

Mistake 2: Forgetting to Check Dimensions First

Always verify that the inner dimensions match before starting any calculation. If A is 3×4 and B is 3×2, you cannot compute A×B. No amount of careful arithmetic will save you if the sizes are incompatible.

Mistake 3: Assuming AB = BA

Commutativity is a habit from ordinary arithmetic that must be completely unlearned for matrices. AB and BA are generally different. Always write matrices in the correct order as required by the problem.

Mistake 4: Getting Row and Column Mixed Up When Reading B

When taking the dot product with a column of B, beginners sometimes accidentally read a row instead. Remember: for the entry C[i][j], you want row i of A and column j of B. Columns run vertically — read down, not across.

Mistake 5: Forgetting to Add — Doing Only the Multiplications

The dot product requires you to multiply and then add. Some beginners carefully multiply all the pairs but then forget to sum them together. The final step — summing all the products — is what gives you the single number that fills one cell of the result.

11. What Does Matrix Multiplication Actually Do?

Understanding the mechanics is great, but understanding the meaning is even better. Here are three powerful ways to think about what matrix multiplication really represents.

Interpretation 1: Chaining Transformations

Each matrix represents a transformation — a rule for taking a vector and producing a new vector. When you multiply A×B, you are creating a single new transformation that does B first, then A. In computer graphics, this is how you combine a rotation, then a scaling, then a translation into one single operation applied to millions of points.

Example: in a 3D game engine, to transform a character's position, you might chain a rotation matrix (R), a scaling matrix (S), and a translation matrix (T) together: Final = T × R × S. One multiplication by "Final" applies all three transformations at once, in the right order.

Interpretation 2: Weighted Combinations of Columns

Here is another way to see it. When you compute A×B, each column of the result is a linear combination of the columns of A, where the weights come from the corresponding column of B. The first column of the result is: (first entry of B's first column) × (first column of A) + (second entry of B's first column) × (second column of A) + ..., and so on. This view connects matrix multiplication directly to the idea of span from the previous chapter.

Interpretation 3: A Compact Multi-Calculation Tool

Going back to our shop: suppose you want the total revenue on every day from every pricing scheme simultaneously. Instead of running the dot product calculation dozens of times, one matrix multiplication gives you all the answers at once, neatly arranged in the result matrix. Matrix multiplication is efficiency — it packages many related calculations into a single, clean operation.

12. Practice Problems

Work through each of these problems on your own before checking the answers below. Go slowly — precision matters more than speed.

Problem 1 (Easy) — Check Dimensions

For each pair, state whether multiplication is possible. If yes, state the size of the result.

(a) A is 2×3 and B is 3×4

(b) A is 4×2 and B is 4×3

(d) A is 3×3 and B is 3×3

Problem 2 (Medium) — Full Multiplication

Compute A × B:

A = [ 2 0 ] B = [ 1 3 ]

[ 1 3 ] [ 2 0 ]

Problem 3 (Medium) — Different Shapes

Compute A × B:

A = [ 1 0 2 ]

[ 3 1 0 ]

B = [ 2 ]

[ 1 ]

[ 4 ]

Problem 4 (Challenge) — Order Matters

Let A = [[1, 2], [0, 1]] and B = [[3, 0], [1, 2]]. Compute both AB and BA and confirm they give different results.

Answers

Problem 1: (a) Yes — result is 2×4. (b) No — inner 2 ≠ 4. (c) Yes — result is 1×1 (a single number). (d) Yes — result is 3×3.

Problem 2:

C[1][1] = (2×1)+(0×2) = 2 C[1][2] = (2×3)+(0×0) = 6

C[2][1] = (1×1)+(3×2) = 7 C[2][2] = (1×3)+(3×0) = 3

Result = [ 2 6 ]

[ 7 3 ]

Problem 3:

C[1][1] = (1×2)+(0×1)+(2×4) = 2+0+8 = 10

C[2][1] = (3×2)+(1×1)+(0×4) = 6+1+0 = 7

Result = [ 10 ]

[ 7 ]

Problem 4:

AB = [ (1×3+2×1) (1×0+2×2) ] = [ 5 4 ]

[ (0×3+1×1) (0×0+1×2) ] [ 1 2 ]

BA = [ (3×1+0×0) (3×2+0×1) ] = [ 3 6 ]

[ (1×1+2×0) (1×2+2×1) ] [ 1 4 ]

AB ≠ BA — confirmed. Order matters.

13. Chapter Summary — Everything in One Place

Here is a complete recap of every key idea from this chapter, in order.

A matrix is a rectangular grid of numbers described by its size as rows × columns.
The dot product of two same-length lists: multiply matching pairs, then sum everything. It always returns one number.
The multiplication rule: A (m×n) times B (n×p) is only valid when the inner dimensions match. The result has size m×p.
How to find each entry: C[i][j] = dot product of row i of A with column j of B.
Chaining linear systems: if y = Ax is one linear system and z = By is another, the combined system is z = (BA)x. The single matrix C = BA encodes the entire two-step pipeline — this is the deepest reason matrix multiplication is defined the way it is.
Not commutative: AB ≠ BA in general. Order always matters — and in a chain, the matrix applied first sits on the right.
Associative: (AB)C = A(BC). You can regroup freely, which is what lets us pre-combine chain matrices efficiently.
Distributive: A(B+C) = AB + AC.
Identity matrix I: AI = IA = A. Works like multiplying by 1.
Meaning: Matrix multiplication chains transformations, takes weighted combinations of columns, and efficiently packages many related calculations — including entire pipelines of linear systems — into one operation.

References & Further Reading

Khan Academy. "Matrices" unit. khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices

3Blue1Brown. "Essence of Linear Algebra," Chapter 4: Matrix Multiplication. YouTube, 2016.

Gilbert Strang. Introduction to Linear Algebra (5th ed.). Wellesley-Cambridge Press, 2016.

End of Dot Product Blog — Linear Algebra Series

Matrix Multiplication