The Complete Solution to Ax = b

Linear-Algebra tutorial · PySpark.in

A Beginner's Guide to Linear Algebra | Part 6 of 4

The Complete Solution to Ax = b

1. The Problem We Are Solving

We know how to find the nullspace — all solutions to Ax = 0. We also know the column space tells us which b values allow Ax = b to be solved at all. Now we put these two ideas together and answer the most important question in linear algebra:

Given a consistent system Ax = b (one that has at least one solution), what is the complete set of all solutions?

The answer is elegant and comes in two pieces. First, find any single solution to Ax = b — we call this the particular solution xₚ. Second, add to it every vector in the nullspace of A. The total collection xₚ + N(A) is the complete solution.

Complete solution: x = xₚ + xₙ

where Axₚ = b and Axₙ = 0

2. Why This Formula Works

The logic is simple. If Axₚ = b and Axₙ = 0, then:

A(xₚ + xₙ) = Axₚ + Axₙ = b + 0 = b ✓

So every vector of the form xₚ + xₙ is indeed a solution. Conversely, if x* is any solution, then:

A(x* - xₚ) = Ax* - Axₚ = b - b = 0

So x* − xₚ is in the nullspace, meaning x* = xₚ + (something in the nullspace). Every solution is captured by our formula. The complete solution is exactly the right-hand side shift of the nullspace: we take the nullspace and slide it to pass through xₚ instead of the origin.

Geometric picture: the nullspace N(A) is a flat subspace passing through the origin. The complete solution to Ax = b is the same shape — same dimension, same directions — but shifted so that it passes through the particular solution xₚ instead of zero. Mathematicians call this a coset or affine subspace.

3. Finding the Particular Solution xₚ

To find xₚ, we set all free variables equal to zero, and solve for the pivot variables. Setting free variables to zero is the simplest possible choice — it gives one particular solution without any extra freedom.

Let us work through a full example. Take the 3×4 augmented system [A | b]:

[ 1 2 2 2 | b₁ ]

[ 2 4 6 8 | b₂ ]

[ 3 6 8 10 | b₃ ]

First, we must check when a solution exists. After elimination (using the same steps as in Blog 5), the augmented matrix becomes:

[ 1 2 2 2 | b₁ ]

[ 0 0 2 4 | b₂-2b₁ ]

[ 0 0 0 0 | b₃-b₂-b₁]

The last row gives: 0 = b₃ − b₂ − b₁. For a solution to exist, the right-hand side must satisfy b₁ + b₂ = b₃. This is the condition for b to be in the column space of A.

Assuming this holds, choose b = (1, 4, 5) (which satisfies 1+4=5 ✓). Set free variables x₂ = 0, x₄ = 0:

From row 2: 2x₃ = 4 - 2(1) = 2 → x₃ = 1

From row 1: x₁ = 1 - 2(0) - 2(1) - 2(0) = -1

Particular solution: xₚ = (-1, 0, 1, 0)

4. The Complete Solution

We already found the nullspace special solutions in Blog 5:

s₁ = (-2, 1, 0, 0) and s₂ = (2, 0, -2, 1)

The complete solution is:

x = xₚ + c₁s₁ + c₂s₂

= (-1, 0, 1, 0) + c₁(-2, 1, 0, 0) + c₂(2, 0, -2, 1)

For any real numbers c₁ and c₂, this gives a valid solution. The solution set is a 2-dimensional affine subspace of R⁴ — a flat plane shifted away from the origin to pass through xₚ.

Complete solution = particular solution + all nullspace solutions. The shape and dimension come from the nullspace. The location (shift from origin) comes from the particular solution.

5. The Four Cases: Rank Determines Everything

Given an m×n matrix A with rank r, there are four possible situations. The rank r determines both the solvability of Ax = b and the uniqueness of solutions.

Case 1: r = m = n — Square, Full Rank (Invertible)

The matrix is square and has a pivot in every row and column. Every b has exactly one solution. The nullspace is just the zero vector. The column space is all of Rᵐ. The matrix A is invertible and x = A⁻¹b.

Example: A = [ 1 2 ], r = 2 = m = n

[ 3 4 ]

Case 2: r = n < m — Full Column Rank

Pivot in every column, but not every row. The nullspace is just {0} — solutions are unique when they exist. But not every b is reachable: the column space is only r-dimensional inside Rᵐ. Either zero solutions (b not in C(A)) or exactly one solution.

Example: A = [ 1 0 ], r = 2 = n, m = 3

[ 0 1 ]

[ 0 0 ]

Case 3: r = m < n — Full Row Rank

Pivot in every row, so Ax = b is solvable for every b — the column space is all of Rᵐ. But there are free variables (n − r of them), so solutions are never unique. Always infinitely many solutions for every b.

Example: A = [ 1 2 3 ], r = 1 = m, n = 3

Case 4: r < m and r < n — Not Full Rank

Both problems at once: not every b is reachable (solvability may fail), and when solutions exist, they are not unique. Either no solution or infinitely many solutions — never exactly one.

Summary: The rank r tells you the dimension of C(A). The number n−r tells you the dimension of N(A). Together they determine whether solutions exist and whether they are unique. This four-case analysis is the master key to the entire subject.

6. Reading the Solution from R

The reduced row echelon form R makes everything explicit. After reducing [A | b] to [R | d], the solution takes a clean form that can be read off directly.

In R, the pivot columns form an identity-like block, and the free columns tell you the special solutions. The d column (the transformed right-hand side) directly gives you the particular solution: the pivot variable values are just the entries of d (with free variables set to zero).

This is why Gauss-Jordan elimination is so powerful — it compresses all the work of finding xₚ and the nullspace basis into a single matrix operation. The result R is the 'answer key' for the whole system.

To solve Ax = b completely: (1) form [A | b], (2) row-reduce to [R | d], (3) check for solvability (no row like [0...0 | c≠0]), (4) read off xₚ by setting free variables to zero, (5) read off special solutions from the free columns of R. Write x = xₚ + c₁s₁ + ... + cₖsₖ.

References & Further Reading

Gilbert Strang. Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, 2016. (Chapter 3.3)

3Blue1Brown. Essence of Linear Algebra, Chapter 7: Inverse Matrices, Column Space, and Null Space. YouTube, 2016.

Khan Academy. Solving Ax = b using the null space. khanacademy.org/math/linear-algebra

End of Blog 6 — Linear Algebra Series