NOTES ON LINEAR ALGEBRA

CONTENTS:

[18] VECTORS AND MATRICES

[19] GENERAL REVIEW

[18] VECTORS AND MATRICES

This section will be a general review on the differences between vectors and matrices. Depending on what problem you’re studying, there are several different ways of looking at a matrix. For this section, we will look at matrices as maps from one Vector Space to another Vector Space.

We won’t go into a technical definition of what a vector space is. Instead, I’ll just mention the ones we’ll be considering: the set of all vectors with exactly two components; the set of all vectors with exactly three components; the set of all vectors with exactly four components; etc.

Now, a vector has magnitude and direction. Let’s take a vector v, and have a matrix A act on it. Now, not every matrix can act on every vector. We have the old row-column rule, which says the number of columns of our Matrix must equal the number of rows of our vector.

Hence

(1 3 5) (4)

(4 6 1) (2)

does not make sense: we get 1*4 + 3*2 + 5*???. However,

(1 3 5) (4)

(4 6 1) (2)

(3 7 9) (3)

(3 4 1)

does make sense.

Let’s consider Av, where our matrix A and v are chosen so that this makes sense. For example, we could have

A = (1 2 3)

(4 5 6)

and

(x)

v = (y)

(z)

Then we find that Av equals

(1x+2y+3z)

(4x+5y+6z)

Note that v is in three-space: it has exactly three components. Av, however, is in two-space: it has exactly two components.

Hence we cannot talk about Av + v. It’s impossible for v to be an eigenvector for A. Why? Let’s say it is an eigenvector, with eigenvalue 2. Then we’d have Av = 2v. The left hand side is a vector with two components. The right hand side is a vector with three components. Trouble!

Think of it as A takes as input vectors with three components, and outputs vectors with just two components. So we cannot talk about Av + v.

This is similar to our troubles with eigenvalue problems. Let’s assume now A is a nice 2x2 matrix, say A equals

(5 5)

(7 3)

and let’s say someone is kind enough to tell us 2 is an eigenvalue, but is unkind enough to ask us to give the corresponding eigenvector. We reason like

Av = 2v

Av - 2v = 0

But we cannot write (A-2)v. Why? A is a 2x2 matrix, whereas 2 is just a number. Hence A - 2 is not defined. What we can do is remember your professor came from an IVy League school.

For any vector v, Iv = v. So 2Iv = 2v.

IMPORTANT NOTE: we are not saying that 2 = 2I. 2 is a number, 2I is a matrix. What we are saying is that the affect of acting on a vector v with the number 2 is the same as the affect of acting on the vector v by the matrix 2I.

Then we get

Av - 2Iv = 0

(A-2I) v = 0

Let’s now quickly review adding vectors. For ease of writing, I’m going to write the vectors horizontally instead of vertically.

So, instead of writing

(1)

(4)

I’ll write (1,4).

Let’s look at 2(1,4). What does this mean? It means we add two copies of (1,4). The answer is (1,4) + (1,4) = (2,8). One adds vectors by adding them componentwise. So, to add two vectors, they must have the same number of components.

3(1,4) = (1,4) + (1,4) + (1,4) = (1+1+1,4+4+4) = (3,12).

More generally, let r be any number. Then

r(1,4) = (1*r, 4*r).

Fractions get a little tricky, but if you remember the above, it should hopefully lessen the confusion. Let’s take, for example,

9/5 (1,4)

Now, if I write 9/5 as 1.8, then it would be

1.8 (1,4) = (1*1.8, 4*1.8) = (1.8, 7.2) = (9/5, 36/5).

When we have a fraction, you just have to remember it’s the fraction times the first component is the new first component; the fraction times the second component is the new second component; etc.

So 9/5 (1,4) = (1 * 9/5, 4 * 9/5) = (9/5, 36/5).

NOT 9/5 (1,4) = (1 * 9, 4 * 5). WRONG!

[19] GENERAL REVIEW

This will be a general review on the differences between matrices, vectors, and numbers. Lots of things that you can do with numbers sadly don’t hold for matrices. However, some things are the same, so it can get a little confusing. Remember, whenever you write something, you need to have a reason justifying it. Being true for numbers is NOT a valid reason.

Let’s look at some examples that are true for numbers and matrices:

1/Addition

3 + 5 = 5 + 3

Or, it doesn’t matter what order you add two numbers.

A + B = B + A

For example,

(1 2) + (3 4) = (4 6) = (3 4) + (1 2)

(5 6) (0 1) (5 7) (0 1) (5 6)

So, you can add two matrices in any order.

You can also add two vectors in any order.

2/Multiplying in a Sequence

Recall what 2 * (3 * 4) means. It means FIRST we multiply 3 and 4, THEN we multiply that by 2 on the left. This is the same as (2 * 3) * 4, which means first multiplying 2 by 3, then multiplying that by 4.

For matrices, it’s the same. A(BC) = (AB)C. However, please not that we do not have A(BC) = (AC)B. And we also don’t have A(BC) = A(CB). We have to keep the matrices in the same order.

Let’s look at some things that are different:

3/Getting Zero:

If m and n are two numbers, and mn = 0, then either m = 0, n = 0, or both m and n equal zero. This is not true for multiplying matrices. For example:

Consider the following product:

(0 1) (1 0)

(0 0) (0 0)

How do we find the first column of the product? It’s just

(0 1) (1) = (0*1 + 1*0) = (0)

(0 0) (0) (0*1 + 0*0) = (0)

How do we find the second column? It’s just

(0 1) (0) = (0*0 + 1*0) = (0)

(0 0) (0) (0*0 + 0*0) = (0)

Hence

(0 1) (1 0) = (0 0)

(0 0) (0 0) (0 0)

So, even though neither matrix is zero, their product is.

4/Switching Order

For numbers, mn = nm – it doesn’t matter which order you multiply them. This, however, is not true for matrices. In general, AB does not equal BA. Let’s do a specific example.

(0 1) (0 0)

(0 0) (1 0)

Then the first column is:

(0 1) (0) = (0*1 + 1*1) = (1)

(0 0) (1) (0*0 + 0*1) (0)

And the second column is:

(0 1) (0) = (0*0 + 1*0) = (0)

(0 0) (0) (0*0 + 0*0) (0)

Hence

(0 1) (0 0) = (1 0)

(0 0) (1 0) (0 0)

Let’s see what happens if you multiply them the other way:

(0 0) (0 1)

(1 0) (0 0)

Then the first column is:

(0 0) (0) = (0*0 + 0*0) = (0)

(1 0) (0) (1*0 + 0*0) (0)

And the second column is:

(0 0) (1) = (0*1 + 0*0) = (0)

(1 0) (0) (1*1 + 0*0) (1)

So we find that

(0 0) (0 1) = (0 0)

(1 0) (0 0) (1 0)

So the two products are not equal!

5/Now, let’s look at what the actions of different

objects on other objects give.

For example, let v be a vector, and consider any number, say 2 for definiteness. Then 2v will also be a vector. It will have the same direction as v, but twice the length. Similarly, -3v will point in the opposite direction from v, and be thrice (is that good Queen’s English?) the length.

Now let’s consider a matrix acting on a vector, say Av. Then this will be a new vector, and except for special v (depending on the matrix A), the direction of Av will not be the same as v. Now, trivially the magnitude of Av will be a multiple of v (think about it – it has to be true!), but as in general their directions are different, it doesn’t help us.

Now let’s go back to the eigenvalue problem. Let’s say we know A, and someone is kind enough to tell you lamda (if a few weeks, you’ll know how to find it yourself). Let’s say lamda is 5. Then we’re trying to solve

Av = 5v

We remember our algebra, which says we want all the unknowns on one side, so we subtract 5v, and get

Av - 5v = 0-vector

Remember: Iv = v. The Identity matrix doesn’t change any vector. Hence

If Iv = v

then 5Iv =5v

So we can substitute for 5v and we go from

Av - 5v = 0-vector

to Av - 5Iv = 0-vector

Now, why did we have to introduce the Identity matrix? We would’ve loved to have been able to go from

Av - 5v to (A-5)v

but alas, we cannot. Why? A is a matrix, 5 is a number, and we cannot subtract a number from a matrix.

Note we’re never saying 5 = 5I – the left hand side is a number, the right hand side is a vector. What we are saying is 5v = 5Iv.

Now we get

(A - 5I)v = 0.

And we can solve this by Gaussian Elimination.

6/Mnemonic for multiplying matrices:

Here’s a way to remember how to multiply matrices:

Say A is

(1 2)

(2 3)

and B is

(1 0)

(2 1)

And we want to find AB. Well, let’s call the first column of B the vector v; let’s call the second column of B the vector u. We know how to multiply

Av, and we know how to multiply Au. Matrix multiplication is just

AB = A(v u) = (Av Au)

This gives the rule write down the first matrix, then write down the first column of the second matrix. That multiplication gives the first colum of the product matrix AB.

So, in our case:

Av = (1 2)(1) = (1*2 + 2*2) = (6)

(2 3)(2) (2*1 + 3*2) (8)

Now we do the first matrix times the second column of B to get the second column of the product matrix AB:

Av = (1 2)(0) = (1*0 + 2*1) = (2)

(2 3)(1) (2*0 + 3*1) (3)