ODE-Project Linear Algebra in a Nutshell

Section 5.1 Linear Algebra in a Nutshell

Objectives

To understand and be able to apply definitions such as linear dependence and independence, basis, and linear combinations to vectors in $R^{2} .$
To understand and be able to do operations such as matrix arithmetic, computing determinants, finding eigenvalues and eigenvectors, and finding matrix inverses for $2 \times 2$ matrices.

Linear algebra and matrices provide a convenient notation for representing the

2 \times 2

system

\begin{aligned} \frac{d x}{d t} & = a x + b y, \\ \frac{d y}{d t} & = c x + d y . \end{aligned}

🔗

If we let

A = (\begin{matrix} a & b \\ c & d \end{matrix}) and x (t) = (\begin{matrix} x (t) \\ y (t) \end{matrix}),

🔗

then we can rewrite our system as

(\begin{matrix} x^{'} (t) \\ y^{'} (t) \end{matrix}) = (\begin{matrix} a x (t) + b y (t) \\ c x (t) + d y (t) \end{matrix}) = (\begin{matrix} a & b \\ c & d \end{matrix}) (\begin{matrix} x (t) \\ y (t) \end{matrix}) .

🔗

In other words, we can write our system as

\frac{d x}{d t} = A x,

🔗

where

x^{'} = \frac{d x}{d t} = (\begin{matrix} x^{'} (t) \\ y^{'} (t) \end{matrix}) .

🔗

Subsection 5.1.1 Matrices and Systems of Linear Equations

🔗

A short review of linear algebra and

2 \times 2

matrices is useful at this point. Recall that any system of two equations in two variables,

\begin{aligned} a x + b y & = α, \\ c x + d y & = β, \end{aligned}

🔗

can be written as a matrix equation

\begin{matrix} (5.1.1) & (\begin{matrix} a & b \\ c & d \end{matrix}) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} a x + b y \\ c x + d y \end{matrix}) = (\begin{matrix} α \\ β \end{matrix}) . \end{matrix}

🔗

We will denote the

2 \times 2

coefficient matrix by

A .

That is,

A = (\begin{matrix} a & b \\ c & d \end{matrix}) .

🔗

If a solution for the system (5.1.1) exists, it is easy to find. A unique solution will occur exactly when the matrix

A

is invertible (or nonsingular). The unique solution is given by

(\begin{matrix} x \\ y \end{matrix}) = {(\begin{matrix} a & b \\ c & d \end{matrix})}^{- 1} (\begin{matrix} α \\ β \end{matrix}),

🔗

where

A^{- 1} = \frac{1}{a d - b c} (\begin{matrix} d & - b \\ - c & a \end{matrix}) .

🔗

The matrix

A

is invertible if and only if its determinant is nonzero,

det (A) = a d - b c \neq 0.

🔗

det (A) = 0,

then we either have no solution or infinitely many solutions.

🔗

Let us consider the special case

A (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .

🔗

det (A) \neq 0,

we have exactly one solution,

x = 0

and

y = 0 .

On the other hand, if

det (A) = 0,

we have infinitely many solutions. Suppose that

a \neq 0 .

Then

x = - (b / a) y,

and

- c (\frac{b}{a}) y + d y = 0.

🔗

Therefore,

(a d - b c) y = 0 .

Since

det (A) = a d - b c = 0,

the variable

y

can assume any value and

x = - (b / a) y .

Thus, the solutions to our system lie along a line through the origin. In fact, we will always get a line of solutions through the origin as long as at least one entry in our matrix is nonzero.

We will not worry about the

2 \times 2

zero matrix, since it will not play a role in our study of linear equations.

🔗

Subsection 5.1.2 Linear Independence

🔗

We say that two vectors

x

and

y

R^{2}

are linearly independent if they do not lie on the same line through the origin. If, on the other hand, they do lie on the same line, then the vectors are linearly dependent. Equivalently, two vectors are linearly dependent if one vector is a multiple of the other. We leave the proof of the following theorem as an exercise.

🔗

Theorem 5.1.1.

🔗

Let

x = (x_{1}, x_{2})

and

y = (y_{1}, y_{2}) .

Then

x

and

y

are linearly independent if and only if

det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.

🔗

If we have a pair of linearly independent vectors in

R^{2},

then we can write any vector in

R^{2}

as a unique linear combination of the two vectors. That is, given two linearly independent vectors

x = (x_{1}, x_{2})

and

y = (y_{1}, y_{2}),

we can write

z = (z_{1}, z_{2})

(\begin{matrix} z_{1} \\ z_{2} \end{matrix}) = α (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) + β (\begin{matrix} y_{1} \\ y_{2} \end{matrix}),

🔗

where

α

and

β

are unique. To see why this is true, we must solve the equations

\begin{aligned} z_{1} & = α x_{1} + β y_{1} \\ z_{2} & = α x_{2} + β y_{2} \end{aligned}

🔗

for

α

and

β .

However, this system has a unique solution since

det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.

🔗

Two vectors are said to be a basis for

R^{2}

if we can write any vector in

R^{2}

as a linear combination of these two vectors. By our arguments above, any two linearly independent vectors will form a basis for

R^{2} .

Example 5.1.2.

The vectors

e_{1} = (1, 0)

and

e_{2} = (0, 1)

form a basis for

R^{2} .

Indeed, if

z = (z_{1}, z_{2}),

then we can write

z = z_{1} e_{1} + z_{2} e_{2} .

The vectors

e_{1}

and

e_{2}

are called the standard basis for

R^{2} .

Example 5.1.3.

Let

v_{1} = (2, 1)

and

v_{2} = (3, 2) .

Since

det (\begin{matrix} 2 & 3 \\ 1 & 2 \end{matrix}) \neq 0,

these vectors form a basis for

R^{2} .

z = (- 5, - 4),

then we can write

z = 2 v_{1} - 3 v_{2} .

We say that the coordinates of

z

are

(2, - 3)

with respect to the basis

{v_{1}, v_{2}} .

Example 5.1.4.

The vectors

(1, 1)

and

(- 1, - 1)

do not form a basis for

R^{2}

since these two vectors lie along the same line.

🔗

2 \times 2

matrices and the rest of what we have described above make you nervous, you should work through the exercises at the end of this section. Below are some examples of matrix operations.

Example 5.1.5.

Compute

A y,

where

A

is the matrix

(\begin{matrix} 2 & 4 \\ - 5 & - 6 \end{matrix})

and

y

is the vector

(\begin{matrix} 10 \\ 1 \end{matrix})

To solve this, we multiply each row of

A

by the vector

y,

and obtain

A y = (\begin{matrix} 2 (10) + 4 (1) \\ - 5 (10) + (- 6) (1) \end{matrix}) = (\begin{matrix} 24 \\ - 56 \end{matrix})

Example 5.1.6.

Compute

A B,

where

A = (\begin{matrix} 2 & 3 \\ - 1 & 5 \end{matrix}), B = (\begin{matrix} - 4 & - 7 \\ 6 & 0 \end{matrix}) .

We multiply each row of

A

by each column of

B,

which gives

A B = (\begin{matrix} 2 (- 4) + 3 (6) & 2 (- 7) + 3 (0) \\ - 1 (- 4) + 5 (6) & (- 1) (- 7) + 5 (0) \end{matrix}) = (\begin{matrix} 10 & - 14 \\ 34 & 7 \end{matrix})

Example 5.1.7.

Compute

A^{- 1},

where

A = (\begin{matrix} 5 & 7 \\ 4 & 6 \end{matrix}) .

Using the formula

{(\begin{matrix} a & b \\ c & d \end{matrix})}^{- 1} = \frac{1}{a d - b c} (\begin{matrix} d & - b \\ - c & a \end{matrix})

we find

A^{- 1} = \frac{1}{5 (6) - 7 (4)} (\begin{matrix} 6 & - 7 \\ - 4 & 5 \end{matrix}) = \frac{1}{2} (\begin{matrix} 6 & - 7 \\ - 4 & 5 \end{matrix}) = (\begin{matrix} 3 & \frac{- 7}{2} \\ - 2 & \frac{5}{2} \end{matrix})

🔗

Activity 5.1.1. Matrix Operations.

🔗

Given the matrices and vectors

A = (\begin{matrix} 5 & 3 \\ - 6 & 4 \end{matrix}), B = (\begin{matrix} 2 & 3 \\ 1 & 2 \end{matrix}), x = (\begin{matrix} 1 \\ 0 \end{matrix}) y = (\begin{matrix} 2 \\ - 1 \end{matrix})

🔗

compute each of the following expressions.

🔗

(a)

🔗

A B,

B A

🔗

(b)

🔗

A^{- 1},

B^{- 1},

(A B)^{- 1},

B^{- 1} A^{- 1}

🔗

(c)

🔗

det (A), det (B), det (A B), det (A^{- 1})

🔗

(d)

🔗

A x,

A y,

y^{T} x,

x y^{T},

where

y^{T} = (2, - 1)

🔗

Subsection 5.1.3 Finding Eigenvalues and Eigenvectors

🔗

A nonzero vector

v

is an eigenvector of

A

A v = λ v

for some

λ \in R .

The constant

λ

is called an eigenvalue of

A .

Letting

A = (\begin{matrix} a & b \\ c & d \end{matrix}) and v = (\begin{matrix} x \\ y \end{matrix}) \neq 0,

🔗

we have

A x = λ v

A v - λ v = 0 .

In matrix form this is

\begin{aligned} (\begin{array}{c} a & b \\ c & d \end{array}) (\begin{array}{c} x \\ y \end{array}) - λ (\begin{array}{c} x \\ y \end{array}) & = (\begin{array}{c} a & b \\ c & d \end{array}) (\begin{array}{c} x \\ y \end{array}) - (\begin{array}{c} λ & 0 \\ 0 & λ \end{array}) (\begin{array}{c} x \\ y \end{array}) \\ = (\begin{array}{c} a - λ & b \\ c & d - λ \end{array}) (\begin{array}{c} x \\ y \end{array}) \\ = (\begin{array}{c} 0 \\ 0 \end{array}) . \end{aligned}

🔗

This matrix equation is certainly true if

(x, y) = (0, 0) .

However, we seek nonzero solutions to this system. This will occur exactly when the determinant of

A - λ I = (\begin{matrix} a - λ & b \\ c & d - λ \end{matrix})

🔗

is zero. In this case

det (A - λ I) = det (\begin{matrix} a - λ & b \\ c & d - λ \end{matrix}) = λ^{2} - (a + d) λ + (a d - b c) .

🔗

We say that

det (A - λ I) = λ^{2} - (a + d) λ + (a d - b c)

🔗

is the characteristic polynomial of

A .

We summarize the results of this discussion in the following theorem.

🔗

Theorem 5.1.8.

🔗

The roots of the characteristic polynomial of

A

are the eigenvalues of

A .

Example 5.1.9.

Suppose that we wish to find the eigenvalues and associated eigenvectors of

A = (\begin{matrix} 1 & 2 \\ 4 & 3 \end{matrix}) .

To find the eigenvalues and eigenvectors for

A,

we must solve the equation

A (\begin{matrix} x \\ y \end{matrix}) = λ (\begin{matrix} x \\ y \end{matrix}) .

If we let

I

denote the

2 \times 2

identity matrix,

I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}),

we can rewrite this equation in the form

\begin{matrix} (5.1.2) & (A - λ I) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) . \end{matrix}

We know that

A - λ I

is a

2 \times 2

matrix and that this system will only have nonzero solutions if

det (A - λ I) = 0 .

In our example,

\begin{aligned} det (A - λ I) & = det (\begin{array}{c} 1 - λ & 2 \\ 4 & 3 - λ \end{array}) \\ = (1 - λ) (3 - λ) - 8 \\ = λ^{2} - 4 λ - 5 \\ = (λ - 5) (λ + 1) . \end{aligned}

Thus,

λ = 5

- 1 .

To see this from a different perspective, we will rewrite equation (5.1.2) as

\begin{aligned} x + 2 y & = λ x \\ 4 x + 3 y & = λ y . \end{aligned}

This system is equivalent to

\begin{aligned} (1 - λ) x + 2 y & = 0 \\ 4 x + (3 - λ) y & = 0 \end{aligned}

which can be reduced to

\begin{aligned} (1 - λ) x + 2 y & = 0 \\ (λ^{2} - 4 λ - 5) y & = 0. \end{aligned}

Therefore, either

λ = 5

λ = - 1

to obtain a nonzero solution.

If $λ = 5,$ the first equation in the system becomes $- 2 x + y = 0,$ and the eigenvectors corresponding to this eigenvalue are the nonzero solutions of this equation. That is, a vector must be a nonzero multiple of $(1, 2)$ to be an eigenvector of $A$ corresponding to $λ = 5 .$
If $λ = - 1,$ then the corresponding eigenvectors are the nonzero multiples of $(1, - 1) .$

🔗

There is also a short-cut to finding the eigenvectors of a

2 \times 2

matrix once you have the eigenvalues, illustrated in the following example.

Example 5.1.10.

Find the eigenvalues of the matrix

A

below, and for each eigenvalue, find an associated eigenvector.

A = (\begin{matrix} 1 & 10 \\ 12 & 8 \end{matrix})

The characteristic polynomial is

(1 - λ) (8 - λ) - 10 (12) = λ^{2} - 9 λ - 112;

setting this equal to zero and solving gives

λ = - 7, 16 .

To find the eigenvectors, we look at each eigenvalue individually and subtract from the magin diagonal; let’s start with

λ = - 7 .

We get

A - (- 7) I = (\begin{matrix} 1 - (- 7) & 10 \\ 12 & 8 - (- 7) \end{matrix}) = (\begin{matrix} 8 & 10 \\ 12 & 15 \end{matrix})

We can "read off" an eigenvector as follows: take one of the rows of the new matrix and put them into a column, flip the numbers around, and change exactly one of the signs. For example, we could take the top row

(\begin{matrix} 8 & 10 \end{matrix})

and find that

(\begin{matrix} - 10 \\ 8 \end{matrix})

and

(\begin{matrix} 10 \\ - 8 \end{matrix})

are value eigenvectors. Likewise, we could have used the bottom row

(\begin{matrix} 10 & 12 \end{matrix})

and gotten

(\begin{matrix} - 12 \\ 10 \end{matrix})

and

(\begin{matrix} 12 \\ - 10 \end{matrix}) .

Now, working with

λ = 16,

we see

A - 16 I = (\begin{matrix} 1 - 16 & 10 / / 12 & 8 - 16 \end{matrix}) = (\begin{matrix} - 15 & 10 \\ 12 & - 8 \end{matrix})

Flipping a row and changing one sign could give us

(\begin{matrix} - 10 \\ - 15 \end{matrix}), (\begin{matrix} 10 \\ 15 \end{matrix}), (\begin{matrix} 8 \\ 12 \end{matrix}),

(\begin{matrix} - 8 \\ - 12 \end{matrix})

as eigenvectors.

🔗

Activity 5.1.2. Finding Eigenvalues and Eigenvectors.

🔗

For each of the following matrices (1) find the characteristic polynomial, (2) find all of the eigenvalues, and (3) find an eigenvector for each eigenvalue.

🔗

(a)

🔗

A = (\begin{matrix} 1 & 3 \\ 1 & - 1 \end{matrix}) .

🔗

(b)

🔗

A = (\begin{matrix} - 8 & 2 \\ - 15 & 3 \end{matrix}) .

🔗

(c)

🔗

A = (\begin{matrix} 4 & 3 \\ - 6 & - 5 \end{matrix}) .

🔗

(d)

🔗

A = (\begin{matrix} 7 & 4 \\ - 10 & - 5 \end{matrix}) .

🔗

(e)

🔗

A = (\begin{matrix} 3 & 1 \\ - 1 & 1 \end{matrix}) .

🔗

Subsection 5.1.4 Important Lessons

🔗

A matrix $A$ is invertible (or nonsingular) if there exists a matrix $A^{- 1}$ such that $A A^{- 1} = A^{- 1} A = I,$ where $I$ is the identity matrix. In the case of $2 \times 2$ matrices,

$I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}) .$
If

$A = (\begin{matrix} a & b \\ c & d \end{matrix}),$

then

$A^{- 1} = \frac{1}{a d - b c} (\begin{matrix} d & - b \\ - c & a \end{matrix}) .$
A matrix $A$ is invertible if and only if its determinant is nonzero,

$det (A) = a d - b c \neq 0.$
We say that two vectors $x$ and $y$ in $R^{2}$ are linearly independent if they do not lie on the same line through the origin. If, on the other hand, they do lie on the same line, then the vectors are linearly dependent. Equivalently, two vectors are linearly dependent if one vector is a multiple of the other.
Let $x = (x_{1}, x_{2})$ and $y = (y_{1}, y_{2}) .$ Then $x$ and $y$ are linearly independent if and only if

$det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.$
If we have a pair of linearly independent vectors in $R^{2},$ then we can write any vector in $R^{2}$ as a unique linear combination of the two vectors. That is, given two linearly independent vectors $x = (x_{1}, x_{2})$ and $y = (y_{1}, y_{2}),$ we can write $z = (z_{1}, z_{2})$ as

$(\begin{matrix} z_{1} \\ z_{2} \end{matrix}) = α (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) + β (\begin{matrix} y_{1} \\ y_{2} \end{matrix}),$

where $α$ and $β$ are unique.
Two vectors are a basis for $R^{2}$ if we can write any vector in $R^{2}$ as a linear combination of these two vectors. Any two linearly independent vectors will form a basis for $R^{2} .$
The roots of the characteristic polynomial, $det (A - λ I),$ of a matrix $A$ are the eigenvalues of $A .$ Given a specific eigenvalue, $λ,$ for a matrix $A,$ the eigenvectors associated with $A$ are the nonzero solutions of the system of equations

$(A - λ I) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .$
If $v_{1}$ and $v_{2}$ are eigenvectors of two distinct real eigenvalues of a matrix $A,$ then $v_{1}$ and $v_{2}$ are linearly independent.