The Timpson Lyceum · Linear Algebra (2026)

4.9 Exercises

Problem sheet · 18 points · 3 extra credit

Applications to Markov Chains

1. 1. (3 pts) Determine whether each matrix is a stochastic matrix. Justify each answer. $A = \begin{bmatrix} 0.6 & 0.3 \\ 0.4 & 0.7 \end{bmatrix}, \qquad B = \begin{bmatrix} 0.9 & 0.5 \\ 0.2 & 0.5 \end{bmatrix}.$

2. 2. (3 pts) For the stochastic matrix $A$ of Problem 1 and initial state $\mathbf{x}_0 = (1, 0)$, compute $\mathbf{x}_1 = A\mathbf{x}_0$ and $\mathbf{x}_2 = A\mathbf{x}_1$.

3. 3. (3 pts) Find the steady-state vector for the stochastic matrix $A = \begin{bmatrix} 0.6 & 0.3 \\ 0.4 & 0.7 \end{bmatrix}$.

4. 4. (3 pts) Find the steady-state vector for the stochastic matrix $P = \begin{bmatrix} 0.5 & 0.2 & 0.3 \\ 0.3 & 0.8 & 0.3 \\ 0.2 & 0.0 & 0.4 \end{bmatrix}.$

5. 5. (3 pts) State whether each of the following is true or false, and justify each answer.

   (a) In a stochastic matrix, the entries of each column sum to $1$.

   (b) A steady-state vector must be a probability vector.

   (c) If $P$ is a regular stochastic matrix, then the state vectors $\mathbf{x}_k$ converge to the unique steady-state vector regardless of the initial state $\mathbf{x}_0$.

   (d) A matrix with a negative entry can still be stochastic.

6. 6. (3 pts) Prove that if $P$ is a stochastic matrix and $\mathbf{x}$ is a probability vector, then $P\mathbf{x}$ is also a probability vector.

7. 7. (3 pts ••) Prove that for every stochastic matrix $P$, the equation $P\mathbf{x} = \mathbf{x}$ has a nontrivial solution — i.e. $P$ has a nonzero fixed vector. (Use only the tools of Chapters 1–4.)