The Timpson Lyceum · Linear Algebra (2026)

Vector Equations and the Matrix Equation Ax = b – Lecture 2 (Linear Algebra)

Overview

The previous lecture solved linear systems row by row. This lecture looks at the same systems column by column, treating each column of a matrix as a vector. It begins with the formal definition of a real vector space (a set with an addition and a scalar multiplication obeying ten axioms) and confirms that $\mathbb{R}^2$ , and more generally $\mathbb{R}^n$ , fit that definition. From there it builds linear combinations and the span of a set of vectors, and shows that asking whether $\mathbf{b}$ is a linear combination of given vectors is the same as asking whether a linear system is consistent. Finally it introduces the matrix equation $A\mathbf{x} = \mathbf{b}$ . That gives four equivalent views of a single system: a list of equations, a vector equation, a matrix equation, and an augmented matrix. The lecture closes with the theorem that decides when $A\mathbf{x} = \mathbf{b}$ is solvable for every $\mathbf{b}$ , and with the proof that multiplication by a matrix is linear.

Key Concepts

Notation

$\mathbb{R}$ is the set of all real numbers. $\forall$ means "for all", $\exists$ means "there exists", and $\in$ means "is an element of". $\mathbb{Z}^+$ is the set of positive integers $1, 2, 3, \dots$
$\mathbb{R}^2 = \mathbb{R} \times \mathbb{R}$ is the set of all ordered pairs of real numbers, and $\mathbb{R}^n$ is the set of all ordered lists of $n$ real numbers, written as columns.
A vector in $\mathbb{R}^n$ is a column. The pair $(1, 13)$ means the column $\begin{bmatrix} 1 \\ 13 \end{bmatrix}$ , which is a different object from the row $\begin{bmatrix} 1 & 13 \end{bmatrix}$ . Order matters, so $(1, 2)$ and $(2, 1)$ are different vectors.

Real vector spaces

A real vector space is a set $V$ $V$ , whose elements are called vectors, together with a vector addition $\oplus$ $\oplus$ and a scalar multiplication $\odot$ $⊙$ , such that for all $\mathbf{u}, \mathbf{v}, \mathbf{w} \in V$ $u, v, w \in V$ and all scalars $\alpha, \beta \in \mathbb{R}$ $α, β \in R$ the following ten axioms hold:
1. $\mathbf{u} \oplus \mathbf{v} \in V$ (closure under addition)
2. $\mathbf{u} \oplus \mathbf{v} = \mathbf{v} \oplus \mathbf{u}$ (commutativity)
3. $\mathbf{u} \oplus (\mathbf{v} \oplus \mathbf{w}) = (\mathbf{u} \oplus \mathbf{v}) \oplus \mathbf{w}$ (associativity)
4. There is a zero vector $\mathbf{0} \in V$ with $\mathbf{u} \oplus \mathbf{0} = \mathbf{u}$ (additive identity)
5. For each $\mathbf{u}$ there is $-\mathbf{u} \in V$ with $\mathbf{u} \oplus (-\mathbf{u}) = \mathbf{0}$ (additive inverse)
6. $\alpha \odot \mathbf{u} \in V$ (closure under scalar multiplication)
7. $\alpha \odot (\mathbf{u} \oplus \mathbf{v}) = (\alpha \odot \mathbf{u}) \oplus (\alpha \odot \mathbf{v})$
8. $\alpha \odot (\beta \odot \mathbf{u}) = (\alpha\beta) \odot \mathbf{u}$
9. $(\alpha + \beta) \odot \mathbf{u} = (\alpha \odot \mathbf{u}) \oplus (\beta \odot \mathbf{u})$
10. $1 \odot \mathbf{u} = \mathbf{u}$
The definition is abstract on purpose. A vector is any member of a vector space, and the operations can be anything that satisfies the axioms. Once the context is clear, $\oplus$ and $\odot$ are written as ordinary $+$ and juxtaposition.
$\mathbb{R}^n$ is a vector space with componentwise operations:

\begin{aligned} \mathbf{u} + \mathbf{v} &= \begin{bmatrix} a_1 \\ \vdots \\ a_n \end{bmatrix} + \begin{bmatrix} b_1 \\ \vdots \\ b_n \end{bmatrix} = \begin{bmatrix} a_1 + b_1 \\ \vdots \\ a_n + b_n \end{bmatrix} \\ c\,\mathbf{u} &= \begin{bmatrix} ca_1 \\ \vdots \\ ca_n \end{bmatrix} \end{aligned}

Each axiom follows from the matching property of real numbers. For example, vector addition is commutative because addition of real numbers is. The zero vector is $(0, 0, \dots, 0)$ .

Other vector spaces appear later in the course. A standard example is the set of polynomials of degree at most $n$ , on which differentiation turns out to be a linear operation.

Vector arithmetic and geometry

With $\mathbf{u} = (1, -2)$ and $\mathbf{v} = (2, -5)$ : $4\mathbf{u} = (4, -8)$ , $(-3)\mathbf{v} = (-6, 15)$ , and $4\mathbf{u} + (-3)\mathbf{v} = (-2, 7)$ .
Geometrically, a vector in $\mathbb{R}^2$ is an arrow from the origin to the point given by its components.
Addition (tip to tail / parallelogram rule): place the tail of $\mathbf{v}$ at the tip of $\mathbf{u}$ . The sum $\mathbf{u} + \mathbf{v}$ runs from the origin to where $\mathbf{v}$ ends. Placing $\mathbf{u}$ after $\mathbf{v}$ gives the same arrow, which is commutativity seen geometrically. For example, $(2, 2) + (-6, 1) = (-4, 3)$ .
Scaling stretches or shrinks the arrow, and a negative scalar also reverses its direction. For $\mathbf{u} = (3, -1)$ , $2\mathbf{u} = (6, -2)$ is twice as long and $-\tfrac{3}{2}\mathbf{u} = (-\tfrac{9}{2}, \tfrac{3}{2})$ points the opposite way at one and a half times the length. Every scalar multiple of $\mathbf{u}$ lies on one line through the origin.

Linear combinations

Given vectors $\mathbf{v}_1, \dots, \mathbf{v}_p \in \mathbb{R}^n$ and scalars $c_1, \dots, c_p$ , the vector

\mathbf{y} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_p\mathbf{v}_p

is a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_p$ with weights $c_1, \dots, c_p$ .

The vector equation $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$ has the same solution set as the linear system whose augmented matrix is $\begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n & \mathbf{b} \end{bmatrix}$ . So $\mathbf{b}$ is a linear combination of $\mathbf{a}_1, \dots, \mathbf{a}_n$ exactly when that system is consistent. Row operations preserve the solution set, so row reducing this matrix answers the vector question directly.

Span

$\operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ is the set of all linear combinations of $\mathbf{v}_1, \dots, \mathbf{v}_p$ . Asking whether $\mathbf{b}$ is in the span is the same as asking whether $x_1\mathbf{v}_1 + \cdots + x_p\mathbf{v}_p = \mathbf{b}$ has a solution.
In $\mathbb{R}^2$ : $\mathbf{v}_1 = (1, 0)$ and $\mathbf{v}_2 = (0, 1)$ span all of $\mathbb{R}^2$ , since $(a, b) = a\mathbf{v}_1 + b\mathbf{v}_2$ . In general, any two vectors in $\mathbb{R}^2$ that are not scalar multiples of each other span $\mathbb{R}^2$ . But $(1, 0)$ and $(3, 0)$ lie on the same line, so every combination of them stays on the $x$ -axis.
In $\mathbb{R}^3$ : the span of one nonzero vector is a line through the origin. The span of two vectors that are not on a common line is a plane through the origin. The span of the zero vector alone is the single point $\{\mathbf{0}\}$ .
A plane through the origin is a small part of $\mathbb{R}^3$ , so a given $\mathbf{b}$ may or may not lie in the span of two vectors. The only way to know is to check whether the system is consistent.

The matrix equation $A\mathbf{x} = \mathbf{b}$

If $A$ is an $m \times n$ matrix ( $m$ rows, $n$ columns) with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ , and $\mathbf{x} \in \mathbb{R}^n$ , then

A\mathbf{x} = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix} \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n

$A\mathbf{x}$ is defined only when $\mathbf{x}$ has as many entries as $A$ has columns. The result has as many entries as $A$ has rows, so $\mathbf{x} \in \mathbb{R}^n$ and $A\mathbf{x} \in \mathbb{R}^m$ . A matrix takes in a vector and returns a vector. This idea will later make matrices behave like functions (linear transformations).
Example: $\begin{bmatrix} 1 & 2 & -1 \\ 0 & -5 & 3 \end{bmatrix}\begin{bmatrix} 4 \\ 3 \\ 7 \end{bmatrix} = 4\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 3\begin{bmatrix} 2 \\ -5 \end{bmatrix} + 7\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 3 \\ 6 \end{bmatrix}$ .
Row equivalence is not equality. Writing $\sim$ between augmented matrices means they have the same solution set. Two matrices are equal only when every entry matches in every position.

Four views of one linear system. The system

\begin{aligned} x_1 + 2x_2 - x_3 &= 4 \\ -5x_2 + 3x_3 &= 1 \end{aligned}

can be written as the vector equation $x_1\begin{bmatrix} 1 \\ 0 \end{bmatrix} + x_2\begin{bmatrix} 2 \\ -5 \end{bmatrix} + x_3\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}$ , as the matrix equation $\begin{bmatrix} 1 & 2 & -1 \\ 0 & -5 & 3 \end{bmatrix}\mathbf{x} = \begin{bmatrix} 4 \\ 1 \end{bmatrix}$ , or as the augmented matrix $\left[\begin{array}{ccc|c} 1 & 2 & -1 & 4 \\ 0 & -5 & 3 & 1 \end{array}\right]$ . All four forms have exactly the same solutions. The augmented matrix is best for computing, and the matrix equation is the form used in most proofs.

Theorem 3. If $A$ is an $m \times n$ matrix with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ and $\mathbf{b} \in \mathbb{R}^m$ , then the matrix equation $A\mathbf{x} = \mathbf{b}$ , the vector equation $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$ , and the system with augmented matrix $\begin{bmatrix} \mathbf{a}_1 & \cdots & \mathbf{a}_n & \mathbf{b} \end{bmatrix}$ all have the same solution set. In particular, $A\mathbf{x} = \mathbf{b}$ has a solution if and only if $\mathbf{b}$ is a linear combination of the columns of $A$ .

Theorem 4. For an $m \times n$ matrix $A$ , the following statements are logically equivalent. They are all true or all false.

For each $\mathbf{b} \in \mathbb{R}^m$ , the equation $A\mathbf{x} = \mathbf{b}$ has a solution.
Each $\mathbf{b} \in \mathbb{R}^m$ is a linear combination of the columns of $A$ .
The columns of $A$ span $\mathbb{R}^m$ .
$A$ has a pivot position in every row.

The last condition concerns the coefficient matrix $A$ , not the augmented matrix. If $A$ has a row with no pivot, some choice of $\mathbf{b}$ produces a row reading $0 = \text{nonzero}$ .

Row–vector rule for computing $A\mathbf{x}$ . Entry $i$ of $A\mathbf{x}$ is the sum of the products of the entries in row $i$ of $A$ with the matching entries of $\mathbf{x}$ :

\begin{bmatrix} 1 & 4 & 7 \\ 2 & 5 & 8 \\ 3 & 6 & 9 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} x_1 + 4x_2 + 7x_3 \\ 2x_1 + 5x_2 + 8x_3 \\ 3x_1 + 6x_2 + 9x_3 \end{bmatrix}

For example, $\begin{bmatrix} 2 & -3 \\ 8 & 0 \\ -5 & 2 \end{bmatrix}\begin{bmatrix} 4 \\ 7 \end{bmatrix} = \begin{bmatrix} 8 - 21 \\ 32 + 0 \\ -20 + 14 \end{bmatrix} = \begin{bmatrix} -13 \\ 32 \\ -6 \end{bmatrix}$ . A square matrix with $1$ s on the main diagonal and $0$ s everywhere else is an identity matrix $I$ , and $I\mathbf{x} = \mathbf{x}$ for every $\mathbf{x}$ .

Theorem 5 (linearity). If $A$ is an $m \times n$ matrix, $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$ , and $c$ is a scalar, then

\begin{aligned} A(\mathbf{u} + \mathbf{v}) &= A\mathbf{u} + A\mathbf{v} \\ A(c\mathbf{u}) &= c(A\mathbf{u}) \end{aligned}

These are the same two conditions that make any operation linear, such as $f(x + y) = f(x) + f(y)$ and $f(cx) = cf(x)$ , or the linearity of expected value in probability.

Worked Example

Is $\mathbf{b}$ a linear combination of $\mathbf{a}_1$ and $\mathbf{a}_2$ ? Let

\mathbf{a}_1 = \begin{bmatrix} 1 \\ -2 \\ -5 \end{bmatrix}, \quad \mathbf{a}_2 = \begin{bmatrix} 2 \\ 5 \\ 6 \end{bmatrix}, \quad \mathbf{b} = \begin{bmatrix} 7 \\ 4 \\ -3 \end{bmatrix}

We need weights with $x_1\mathbf{a}_1 + x_2\mathbf{a}_2 = \mathbf{b}$ . That is the system $x_1 + 2x_2 = 7$ , $-2x_1 + 5x_2 = 4$ , $-5x_1 + 6x_2 = -3$ , so row reduce its augmented matrix:

\begin{aligned} \left[\begin{array}{cc|c} 1 & 2 & 7 \\ -2 & 5 & 4 \\ -5 & 6 & -3 \end{array}\right] &\sim \left[\begin{array}{cc|c} 1 & 2 & 7 \\ 0 & 9 & 18 \\ 0 & 16 & 32 \end{array}\right] && R_2 \to R_2 + 2R_1,\; R_3 \to R_3 + 5R_1 \\ &\sim \left[\begin{array}{cc|c} 1 & 2 & 7 \\ 0 & 1 & 2 \\ 0 & 16 & 32 \end{array}\right] && R_2 \to \tfrac{1}{9} R_2 \\ &\sim \left[\begin{array}{cc|c} 1 & 2 & 7 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{array}\right] && R_3 \to R_3 - 16R_2 \\ &\sim \left[\begin{array}{cc|c} 1 & 0 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{array}\right] && R_1 \to R_1 - 2R_2 \end{aligned}

The system is consistent with $x_1 = 3$ and $x_2 = 2$ . The zero row shows that one of the three equations added no new information. Check:

3\mathbf{a}_1 + 2\mathbf{a}_2 = \begin{bmatrix} 3 \\ -6 \\ -15 \end{bmatrix} + \begin{bmatrix} 4 \\ 10 \\ 12 \end{bmatrix} = \begin{bmatrix} 7 \\ 4 \\ -3 \end{bmatrix} = \mathbf{b}

So $\mathbf{b} \in \operatorname{Span}\{\mathbf{a}_1, \mathbf{a}_2\}$ .

When is $A\mathbf{x} = \mathbf{b}$ consistent for every $\mathbf{b}$ ? Take $A = \begin{bmatrix} 1 & 3 & 4 \\ -4 & 2 & -6 \\ -3 & -2 & -7 \end{bmatrix}$ and a general $\mathbf{b} = (b_1, b_2, b_3)$ :

\begin{aligned} \left[\begin{array}{ccc|c} 1 & 3 & 4 & b_1 \\ -4 & 2 & -6 & b_2 \\ -3 & -2 & -7 & b_3 \end{array}\right] &\sim \left[\begin{array}{ccc|c} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & b_2 + 4b_1 \\ 0 & 7 & 5 & b_3 + 3b_1 \end{array}\right] && R_2 \to R_2 + 4R_1,\; R_3 \to R_3 + 3R_1 \\ &\sim \left[\begin{array}{ccc|c} 1 & 3 & 4 & b_1 \\ 0 & 14 & 10 & b_2 + 4b_1 \\ 0 & 0 & 0 & b_1 - \tfrac{1}{2}b_2 + b_3 \end{array}\right] && R_3 \to R_3 - \tfrac{1}{2}R_2 \end{aligned}

The third row of $A$ has no pivot, so by Theorem 4 the equation is not solvable for every $\mathbf{b}$ . It is consistent exactly when $b_1 - \tfrac{1}{2}b_2 + b_3 = 0$ , which is the equation of a plane through the origin in $\mathbb{R}^3$ . The three columns of $A$ span only that plane, because one column is a linear combination of the other two.

Summary

A real vector space is a set with an addition and a scalar multiplication satisfying ten axioms. A vector is any element of such a set, and $\mathbb{R}^n$ with componentwise operations is the main example.
Geometrically, vectors add tip to tail (the parallelogram rule), and scalar multiplication stretches, shrinks, or reverses an arrow along its own line.
$\operatorname{Span}\{\mathbf{v}_1, \dots, \mathbf{v}_p\}$ is the set of all linear combinations. Testing whether $\mathbf{b}$ is in a span means testing whether the augmented matrix $\begin{bmatrix} \mathbf{v}_1 & \cdots & \mathbf{v}_p & \mathbf{b} \end{bmatrix}$ is consistent.
$A\mathbf{x}$ is the linear combination of the columns of $A$ weighted by the entries of $\mathbf{x}$ . For an $m \times n$ matrix it takes $\mathbf{x} \in \mathbb{R}^n$ to $A\mathbf{x} \in \mathbb{R}^m$ , and the row–vector rule computes it quickly.
A linear system can be written as equations, a vector equation, a matrix equation, or an augmented matrix, and all four have the same solutions.
$A\mathbf{x} = \mathbf{b}$ is solvable for every $\mathbf{b} \in \mathbb{R}^m$ exactly when the columns of $A$ span $\mathbb{R}^m$ , which happens exactly when $A$ has a pivot in every row. Multiplication by $A$ is linear: $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$ and $A(c\mathbf{u}) = c(A\mathbf{u})$ .