Matrix algebra

Section 3 Matrix algebra

Subsection 3.1 Matrix notation

A real matrix is a rectangular array of real numbers, called the entries or elements in the matrix. Matrix entries are indexed by a pair of integers $i,j$ that indicate the row and column, respectively, of the entry. Rows are counted from top to bottom and columns are counted from left to right. A matrix with $m$ rows and $n$ columns is called an $m\times n$ matrix (pronounced "$m$ by $n$ matrix"). We write $[a_{ij}]$ to denote a matrix with entries $a_{ij}$ (here, we adopt the common practice of omitting the comma between the subscripted indices $i,j$).

\begin{equation} [a_{ij}] = \left[\begin{array}{cccccc} a_{11} \amp a_{12} \amp \cdots \amp a_{1j}\amp \cdots \amp a_{1n}\\ a_{21} \amp a_{22} \amp \cdots \amp a_{2j} \amp \cdots \amp a_{2n}\\ \vdots \amp \vdots \amp \amp \vdots \amp \amp \vdots\\ a_{i1} \amp a_{i2} \amp \cdots \amp a_{ij}\amp\cdots \amp a_{in}\\ \vdots \amp \vdots \amp \amp\vdots \amp \amp \vdots\\ a_{m1} \amp a_{m2} \amp \cdots \amp a_{mj} \amp \cdots\amp a_{mn} \end{array}\right]\tag{3.1} \end{equation}

A matrix with only 1 row is called a row matrix and a matrix with only 1 column is a column matrix. Entries of row and column matrices (the plural of "matrix" is "matrices") are usually given with just one index.

\begin{equation*} \mbox{a row matrix } \left[ \begin{array}{cccc} a_1 \amp a_2 \amp \cdots \amp a_m \end{array}\right], \rule{.5in}{0in}\mbox{ a column matrix } \left[ \begin{array}{c} b_1\\ b_2\\ \vdots \\ b_n \end{array}\right] \end{equation*}

In the matrix (3.1), it is sometimes useful to write

\begin{equation} [a_{ij}]= \left[ \mathbf{c}_1 \; \mathbf{c}_2 \; \cdots \; \mathbf{c}_n\right]\tag{3.2} \end{equation}

where $\mathbf{c}_j = \left[\begin{array}{c}a_{1j}\\ a_{2j}\\ \vdots\\ a_{mj}\end{array}\right]$ is the $j$th column of the matrix $[a_{ij}]\text{.}$

Subsection 3.2 Matrices and vectors

In matrix algebra, vectors are identified with column matrices.

In some texts, $n$-tuples denoted by comma-separated lists are considered to be different types of objects from $n$-tuples written as column matrices. In these notes, we consider the comma-separated list and the column matrix to simply be two notations for the same thing.

\begin{equation*} \mathbf{x}= (x_1,x_2,\ldots,x_n) = \left[\begin{array}{c} x_1\\ x_2\\ \vdots\\ x_n \end{array}\right] \end{equation*}

Subsection 3.3 Matrices and linear transformations

Next, we establish a one-to-one correspondence between linear transformations $\R^n\to \R^m$ and $m\times n$ matrices.

Given a linear transformation $L\colon \R^n\to \R^m\text{,}$ let $[L]$ denote the $m\times n$ matrix whose columns are the vectors $L\mathbf{e}_1,L\mathbf{e}_2,\ldots,L\mathbf{e}_n\text{.}$

\begin{equation} [L]= \begin{bmatrix} L\mathbf{e}_1 \amp L\mathbf{e}_2 \amp \cdots \amp L\mathbf{e}_n \end{bmatrix}.\tag{3.3} \end{equation}

Conversely, given an $m\times n$ matrix $A\text{,}$ let $L_A$ denote the linear transformation $\R^n\to \R^m$ given by

\begin{equation*} L_A(x_1,x_2,\ldots,x_n) = \sum_{j=1}^n x_j(\mbox{$j$th column of $A$}). \end{equation*}

It is clear that

\begin{equation*} L_A\mathbf{e}_j = (\mbox{$j$th column of $A$}) \end{equation*}

and therefore, that $[L_A] = A\text{.}$ We conclude that the correspondence $L\leftrightarrow [L]$ is a one-to-one correspondence between the set of linear transformations $\R^n\to \R^m$ and the set of $m\times n$ matrices.

\begin{equation} \begin{array}{ccc} \{\mbox{linear maps } \R^n\to \R^m\} \amp\longleftrightarrow \amp \{m\times n\mbox{ matrices}\}\\ L \amp\longleftrightarrow \amp [L] \end{array}\tag{3.4} \end{equation}

Subsection 3.4 Matrix algebra

Using the one-to-one correspondence $L\leftrightarrow [L]\text{,}$ we can define the following matrix operations. Let $A$ be an $m\times n$ matrix, and let $\mathbf{x}$ be an $n\times 1$ column matrix. By (3.4), there is a linear transformation $L\colon \R^n\to \R^m$ such that $[L]=A\text{.}$ We define $A\mathbf{x}$ to be equal to $L\mathbf{x}\text{.}$ Given a scalar $\alpha\text{,}$ we define $\alpha A$ to be equal to $[\alpha L]\text{.}$ If $A'$ is another $m\times n$ matrix with $A'=[L']\text{,}$ we define $A+A'$ to be equal to $[L+L']\text{.}$ Finally, if $B$ is an $n\times p$ matrix such that $B=[M]\text{,}$ we define $AB$ to be equal to $[LM]\text{.}$

Here is a summary of the operations of matrix algebra.

\begin{align} [L]\mathbf{x}\amp=L\mathbf{x}\amp \text{(matrix times a vector)}\tag{3.5}\\ \alpha [L] \amp= [\alpha L]\amp \text{(matrix scalar multiplication)}\tag{3.6}\\ [L]+[L'] \amp= [ L+L'] \amp \text{(matrix addition)}\tag{3.7}\\ [L][M] \amp= [LM]\amp \text{(matrix multiplication)}\tag{3.8} \end{align}

Here are formulas for matrix operations in terms of matrix entries.

\begin{align} \mbox{$i,j$ entry of $\alpha A$} \amp= \alpha \mbox{($i,j$ entry of $A$)} \tag{3.9}\\ \mbox{$i,j$ entry of $A+A'$} \amp=\mbox{($i,j$ entry of $A$)} + \mbox{($i,j$ entry of $A'$)}\tag{3.10}\\ \mbox{$i,j$ entry of $AB$} \amp= \mbox{($i$th row of $A$)} \cdot \mbox{($j$th column of $B$)}\tag{3.11} \end{align}

In most texts, the above formulas are given as definitions. In these notes, these formulas are consequences of the definitions (3.6), (3.7), and (3.8). We make this choice to emphasize that matrix algebra operations are natural because they come from the corresponding natural operations on linear maps. There is a fourth basic operation, called transposition , whose corresponding operation on linear maps is less easy to describe. We define the transpose of matrix $A\text{,}$ denoted $A^T$, by

\begin{equation} \mbox{$i,j$ entry of $A^T$} = \mbox{$j,i$ entry of $A$}.\tag{3.12} \end{equation}

Here is how the inner product can be written in terms matrix algebra using transposition.

\begin{equation} \mathbf{x}\cdot \mathbf{y}=\mathbf{x}^T\mathbf{y}\\\tag{3.13} \end{equation}

Exercises 3.5 Exercises

1.

Perform the matrix multiplications below.

\begin{equation*} \left[\begin{array}{cc} 1 \amp -2\\ -1 \amp 3 \end{array}\right] \left[\begin{array}{cc} 0 \amp 2\\ 1 \amp -1 \end{array}\right] \end{equation*}
\begin{equation*} \left[\begin{array}{ccc} 1 \amp -2 \amp 0\\ -1 \amp 3 \amp 1 \end{array}\right] \left[\begin{array}{cc} 0 \amp 2\\ 1 \amp -1\\ 2 \amp 0 \end{array}\right] \end{equation*}

Answer.

$\displaystyle \left[\begin{array}{cc} -2 \amp 4\\ 3 \amp -5 \end{array}\right]$
$\displaystyle \left[\begin{array}{cc} -2 \amp 4\\ 5 \amp -5 \end{array}\right]$

2.

Let $L\colon \R^3\to\R^2$ be a linear map with $L\mathbf{e}_1=(1,2)\text{,}$ $L\mathbf{e}_2=(-1,1)\text{,}$ and $L\mathbf{e}_3=(0,1)\text{.}$

Write the matrix for L.
Evaluate $L(2,1,3)\text{.}$
Evaluate $L(0,1,1)\text{.}$

Answer.

$\displaystyle \left[\begin{array}{ccc} 1 \amp -1 \amp 0\\ 2 \amp 1 \amp 1 \end{array}\right]$
$\displaystyle (1,8)$
$\displaystyle (-1,2)$

3.

Let $L\colon\R^2\to\R^3$ be a linear map with the following matrix.

\begin{equation*} \left[\begin{array}{cc} 1 \amp 2\\ 3 \amp -1\\ 0 \amp 1 \end{array}\right] \end{equation*}

Evaluate $L(1,2)\text{.}$
Evaluate $L(-2,1)\text{.}$

Answer.

$\displaystyle (5,1,2)$
$\displaystyle (0,-7,1)$

4.

Equation (3.11) is a fundamental formula for matrix calculations. This exercise outlines a proof. Let $L,M$ be linear transformations

\begin{equation*} \R^p \stackrel{M}{\to} \R^n \stackrel{L}{\to} \R^m. \end{equation*}

Let $a_{ij}$ be the element in the $i$th row and $j$th column of $[L]\text{,}$ and let $\mathbf{r}_i,\mathbf{c}_j$ denote the $i$th row and $j$th column of $[L]\text{,}$ respectively.

\begin{align*} \mathbf{r}_i \amp = (a_{i1},a_{i2},\ldots,a_{in})\\ \mathbf{c}_j \amp = (a_{1j},a_{2j},\ldots, a_{mj}) \end{align*}

Justify each step in the derivation below.

For $\mathbf{x}$ in $\R^n\text{,}$ we have
\begin{equation*} L\mathbf{x} = x_1\mathbf{c}_1 + x_2\mathbf{c}_2 + \cdots + x_n\mathbf{c}_n = \begin{bmatrix} \mathbf{r}_1 \cdot \mathbf{x}\\ \mathbf{r}_2 \cdot \mathbf{x}\\ \vdots\\ \mathbf{r}_m \cdot \mathbf{x}\\ \end{bmatrix}. \end{equation*}
Now let $\mathbf{x}=M(\mbox{$j$th standard basis vector})$ so that (say why!)
\begin{equation*} \mathbf{x}= (\mbox{$j$th column of $[M]$}). \end{equation*}
We have
\begin{align*} (\mbox{$j$th column of $[LM]$}) \amp= LM(\mbox{$j$th standard basis vector})\\ \amp= L(M(\mbox{$j$th standard basis vector})) \end{align*}
From this we conclude (say how!) that the $i,j$ entry in $[LM]$ is
\begin{equation*} (\mbox{$i$th row of } [L]) \cdot (\mbox{$j$th col. of } [M]). \end{equation*}

5.

Let $L\colon \R^2\to \R$ and $K\colon \R\to\R^3$ be linear maps such that $L{\mathbf i}=1\text{,}$ $L{\mathbf j}=-2$ and $K(1)=(2,1,3)\text{.}$

Write the matrices for $L$ and $K\text{.}$
Find the matrix for $KL\text{.}$
Find $KL(2,-1)\text{.}$

Answer.

$L= \left[\begin{array}{cc} 1 \amp -2 \end{array}\right]\text{,}$ $K = \left[\begin{array}{c} 2\\ 1\\ 3 \end{array}\right]$
$\displaystyle \left[\begin{array}{cc} 2 \amp -4\\ 1 \amp -2\\ 3 \amp -6 \end{array}\right]$
$\displaystyle (8,4,12)$

Prev Top Next