NOTES
ON LINEAR ALGEBRA
CONTENTS:
[1]
MULTIPLYING MATRICES
[2] GAUSSIAN
ELIMINATION
[3] INVERTING
MATRICES
[1] MULTIPLYING MATRICES:
For
ease of presentation, I will NOT draw the parentheses around the matrices
correctly. If I were to, I’d have to use either the Equation Editor (which
takes more time) or LaTeX (which your computer has
trouble reading).
Let’s
say we have the matrix A =
(1 2)
(3 4)
And
we want to multiply it by the column vector v =
(5)
(6)
The
answer is Av =
(1 2) (5) (1*5 + 2*6) (17)
(3 4) (6) = (3*5 +
4*6)
= (39)
Let’s
do another example. Let B =
(2 7)
(3
5)
and
let the vector w =
(1)
(3)
Then
B w =
(2 7) (1) (2*1 + 7*3) (23)
(3
5) (3) = (3*1 + 5*3) = (18)
Let’s
study a bigger matrix now. Let C =
(1 2)
(3 4)
(5 6)
and
consider the vector x =
(2)
(1)
Then
C x =
(1 2) (2) (1*2 + 2*1) ( 4 )
(3 4) (1)
= (3*2 + 4*1) =
(10)
(5 6) (5*2 + 6*1) (16)
And
finally, let’s look at D =
(1 2 0)
(3 4 2)
(5 6 3)
and
the vector y =
(1)
(0)
(2)
Then
the product D y =
(1 2 0)
(1) (1*1
+ 2*0 + 0*2)
(3 4 2) (0) = (3*1 + 4*0 + 2*0)
(5 6 3) (2)
(5*1 + 6*0
+ 3*2)
This
is basically how to multiply a matrix by a column vector. Now we want to study
how to multiply two matrices together. We have the following rule, which we
proved:
Matrix
Multiplication Rule:
Let’s say A has Ra rows and
Ca columns, and B has Rb rows
and Cb columns. This means A is
an Ra x Ca matrix, and B is an
Rb x Cb matrix. Then we can do the multiplication AB if and
only if Ca = Rb, and the
resulting matrix AB has Ra rows and Cb
columns.
For
example, if A is 3x4 and B is 4x2, then we can do the multiplication AB, and
the product AB is a 3x2 matrix; however, we cannot do the multiplication BA,
for 2 ¹ 3.
Let’s
do some examples: Let the matrices A and B be (respectively)
(1 2) and (5 6)
(3 4) (7
8)
Then
in this case we can multiply in EITHER order, as both are 2x2.
Let’s do AB =
(1 2) (5 6)
(3 4) (7 8)
The
way we multiply matrices is column by column. To find the first column in the
product, we multiply the matrix A by the first column of B, and that’s the
answer. To find the second column of the product, we multiply A by the second
column of B.
Step
1: Finding the first column of the product:
(1 2) (5) (1*5 + 2*7) (19)
(3 4) (7) = (3*5 + 4*7) = (43)
Step
2: Finding the second column of the product:
(1 2) (6) (1*6 + 2*8) (22)
(3 4) (8) = (3*6 + 4*8) = (50)
Step
3: Combining the above:
(1 2) (5 6) (19 22)
(3 4) (7 8) = (43 50)
Let’s
do a harder one: Let the matrices C and D be
(respectively)
(1 2 3) (3 0)
(4 5 6) and (1 2)
(2 1 0) (0 5)
First,
let’s check and make sure we can multiply CD. C is 3x3,
D is 3x2, so yes we can, and the product will be 3x2.
Step
1: C times the first column of D gives the first column of CD
(1
2 3) (3) (1*3 +
2*1 + 3*0) ( 5)
(4 5 6) (1) = (4*3 + 5*1 + 6*0) = (17)
(2 1 0) (0) (2*3 + 1*1 + 0*0) ( 7)
Step
2: C times the second column of D gives the second column of CD
(1 2 3) (0) (1*0 + 2*2 + 3*5) (19)
(4 5 6) (2) = (4*0 + 5*2 + 6*5) = (40)
(2 1 0) (5) (2*0 + 1*2 + 0*5) ( 2)
Step
3: Combining the above yields CD =
(1 2 3) (3 0) (5 19)
(4 5 6) (1 2) = (17 40)
(2 1 0) (0 5) ( 7 2 )
[2] GAUSSIAN ELIMINATION:
Matrices
can be used to represent systems of equations, which we then try to solve. For
example, let’s say we have the two equations:
3x + 2y = 5
4x + 5y = 7
Then
we can write this in matrix form by
(3 2) (x) (5)
(4 5) (y) = (7)
Or,
if we had the three equations
3x + 2y + 5z = 8
2x + 2y + 4z = 7
7x + 9y + 0z = 1
Then
we can write this in matrix form by
(3 2 5) (x) (8)
(2 2 4)
(y) = (7)
(7 9 0) (z) (1)
Now,
we want to find a way to solve such systems of equations. Let’s start with an
easy example:
1x + 2y = 1
3x + 7y = 2
We
can write this in matrix form by
(1 2) (x) (1)
(3 7) (y) = (2)
Now,
let’s look at the two equations. If we multiply the first equation by -3 we
get: -3x -6y = -3. If we then add this to the second
equation (3x + 7y = 2) we get a new second equation:
3x + 7y =
2
+ -3x - 6y
= -3
------------------
0x + 1y = -1
So
now we have the two equations
1x + 2y = 1
0x + 1y = -1
which we can write in matrix form as
(1 2) (x) ( 1)
(0 1) (y) = (-1)
We
started with the matrix
(1 2) (x) (1)
(3 7) (y) = (2)
If
we multiply the first row by -3 and add that to the second row, we get the
matrix
(1 2)
(0 1)
And
if we multiply 1 by -3 and add it to 2 we get the vector
( 1)
(-1)
So
we see we can symbolically represent multiplying and adding equations by multiplying
and adding rows. Slowly, here goes:
The goal is to reduce the
matrix to something easy to work with, namely something with all zeros below
the diagonal.
We
start with
(1 2) (x) (1)
(3 7) (y) = (2)
Step
1: What do we need to multiply the first row by to cancel the 3 in the second
row? Or, find ‘a’ such that 1a + 3 = 0, hence a = -3. We then multiply the
first row by -3, and write the result under the second row.
Question 1: why do we multiply the first row by -3? Because that’s what we need to cancel the 3 in the second row.
Question 2: why do we write the result under the
second row? Because
that’s where we’re adding the
result.
Remember,
you must also multiply 1 by -3 and add it to 2. Why? Equality: whatever you do to one side of the equation, you must do to the other.
[-3]
(1 2) (x) (1)
(3 7) (y) = (2)
-3 -6 -3
(1 2) (x) ( 1)
(0 1) (y) = (-1)
Step
2: We can now read off the answers! The two equations are:
1x + 2y = 1
0x + 1y = -1
So
we learn from the second equation that y = -1. We then substitute that value
into the first equation and get 1x + 2(-1) = 1, so x = 3. We can check this by
substituting these values for x and y into the original equations:
1x + 2y = 1 è 1(3) +
2(-1) = 1
3x + 7y = 2 è 3(3) + 7(-1)
= 2
So yes, these values work.
Let’s do a slightly harder example.
Consider the following:
(1 2 3) (x) ( 2)
(2 3 0)
(y) = (
1)
(3 0 1)
(z) (10)
Step 1: we want to get a matrix that has all zeros under the
diagonal. So we need to get rid of the 2 in the second row and the 3 in the
third row. To get rid of the 2 in the second row, we multiply the first row by -2 and add the result to the second row; we multiply the
first row by -3 and add the result
to the third row. Remember, we write the results of the multiplication under
the row we’re going to add it to, and remember we MUST also do the multiplication on the right hand side. So we must
multiply 1 by -2 and we must
multiply 10 by -3.
(1 2 3) (x) ( 2)
[-2] (2 3 0) (y) = ( 1)
-2 -4 -6 -4
[-3] (3 0 1) (z) (10)
-3 -6 -9 -6
This
gives
(1 2
3)(x) (2)
(0 -1 -6)(y) = (-3)
(0 -6 -8)(z) ( 4)
We’re
almost there – we now need to get rid of the -6 in the third row. Then we’ll have
a matrix with all zeros under the main diagonal, and we’ll be able to read off
the answers.
Step
2: We need to get rid of the -6 in the third row. There’s nothing we can
multiply the first row by. Why? If we add copies of the first row to the third,
we’ll lose the 0 which starts off the third row. What we should do is multiply
the second row by something and add it to the third, as this way we won’t lose
the zero. So, we need to find ‘a’ such that (-1)a +
(-6) = 0, hence a = -6.
(1 2
3)(x) (2)
(0 -1 -6)(y) = (-3)
[-6] (0
-6 -8)(z) ( 4)
0 6 36 18
This yields
(1 2
3)(x) ( 2)
(0 -1 -6)(y) = (-3)
(0 0
28)(z) (22)
Step
3: We can now read off the answers! The three equations are
1x + 2y + 3z = 2
0x - 1y
- 6z = -3
0x + 0y +28z = 22
So
z = 22/28 =
11/14
So
-y - 6(11/14) = -3 è y = -24/14
So
x + 2(-24/14) + 3(11/14) = 2 è x = 43/14
Let’s
check these numbers in the original equations:
1x + 2y + 3z = 2 è 1(43/14) + 2(-24/14) + 3(11/14)
= 2
2x + 3y + 0z = 1 è 2(43/14) + 3(-24/14) + 0(11/14)
= 1
3x + 0y + 1z = 10 è 3(43/14) + 0(-24/14) +
1(11/14) = 10
So we see we do obtain the correct answer! (If it makes you
feel better, I got wrong answers the first two times I did the problem – I did
the algebra wrong).
[3] INVERTING MATRICES:
We’re
now ready to use the method of Gaussian Elimination to invert matrices. Let’s
review how Gaussian Elimination works. We start off with a matrix A and we do
row operations to it. This is equivalent to multiplying A by several matrices E1,
E2, ..., En (say).
For
simplicity, let’s assume it takes 5 steps to Gaussian Eliminate A to the
Identity matrix, so E5 E4 E3 E2 E1
A = I. Then E5 E4 E3 E2 E1
= A-1, the inverse matrix to A.
To
keep track of these steps, we can just form E5 E4 E3
E2 E1
I, which by the above is A-1.
An
example should illustrate.
Let’s
try to find the inverse to A =
(1 2)
(3 5)
THE GOAL: We will use Gaussian
Elimination to get A to the identity matrix (ones on the main diagonal, zeros
elsewhere). We will keep track of the Gaussian Elimination by acting on the
Identity matrix.
Step
1: Write the matrix A followed by the identity:
(1 2) (1
0)
(3 5) (0
1)
Step
2: We need to eliminate the 3 in the second row, so we must find ‘a’ such that
1a + 3 = 0. Hence a = -3. So we multiply
the first row of A by -3 and add it to the
second row. And remember, by EQUALITY,
we must do the same to the other side, to the Identity.
(1 2) (1
0)
[-3] (3 5) (0 1)
-3 -6
-3 0
(1
2) (1 0)
(0 -1) (-3
1)
Step
3: Now, we want to have all 1s along the main diagonal, so we might as well
adjust the second row right now. We have a -1, where we want a 1. So we must
multiply the second row by -1. Again, must do this to both sides:
(1
2) (1 0)
(0 -1) (-3
1)
0 1 3 -1
Hence
we get
(1 2) (1 0)
(0 1) (3
-1)
Step
4: Now we need to get rid of the 2 in the first row, so we multiply the second
row by -2 and get:
(1 2) (1 0)
0 -2 -6 2
(0 1) (3
-1)
and
we get
(1 0) (-5
2)
(0 1) ( 3 -1)
Note:
as a check, you can go thru and see that
(-5 2)
( 3 -1)
is
the inverse to A.
Let’s
do one more problem. Let’s find the inverse for B =
(9 4)
(7 3)
Step
1: Write the matrix B followed by the Identity:
(9 4) (1
0)
(7 3) (0
1)
Step
2: What should we multiply the first row by to get rid of the 7 in the second
row? So, find ‘a’ such that 9a + 7 = 0, or a = -7/9.
(9 4) (1 0)
[-7/9] (7 3) (0 1)
-7 -28/9 -7/9 0
And
we get
(9
4) (1 0)
(0 -1/9) (-7/9 1)
Step
3: We want to end up with the identity matrix on the left. We have -1/9 in the
lower diagonal – we need to multiply the second row by -9 to get 1.
(9
4) (1 0)
[-9] (0
-1/9) (-7/9 1)
0
1 7 -9
And
we get
(9 4) (1 0)
(0 1) (7 -9)
Step
4: We need to get rid of the 4 in the first row, so we multiply the second row
by -4 and add it to the first
[-4] (9
4) (1 0)
0 -4 -28 36
(0 1) (7 -9)
And
we get
(9 0) (-27
36)
(0 1) ( 7 -9)
Step
5: We need to have the identity on the left. We have a 9 in the upper left
corner, so we must multiply the first row by 1/9.
[1/9] (9 0) (-27 36)
1 0 -3
4
(0 1) ( 7 -9)
And
we get
(1 0) (-3
4)
(0 1) ( 7 -9)
You
can check that this is the inverse of B by doing the multiplication.