NOTES ON LINEAR ALGEBRA

 

CONTENTS:

[4] MATRIX ADDITION

[5] MATRIX NOTATION

[6] TRANSPOSE

[7] SYMMETRIC MATRICES

[8] BASIC FACTS ABOUT MATRICES

 

 

[4] MATRIX ADDITION

Let A and B be two matrices. When can we add them, and what is the answer? We define matrix addition by adding componentwise. For example:

 

            (1 2)         +     (5 7)       =    (6 9)     

            (3 4)                  (2 0)              (5 4)

 

Or

 

            (1 2 5)     +     (5 7 1)       =    (6 9 6)           

            (3 4 0)               (2 0 8)              (5 4 8)

 

 

 

Of course, we’ve yet to give any motivation as to why one would want to define matrix addition by the above. Remember how we introduced matrices as maps from one space to another. For example, consider the matrix

 

            (1 2 5)

            (3 4 0)

 

It has 2 rows and 3 columns. It acts on vectors with three components, and returns something with 2 components. For example:

 

            (1 2 5)  (3)       (1*3 + 2*2 + 5*1)      ( 7)

            (3 4 0)  (2)   =  (3*3 + 4*2 + 0*1)  =  (17)

                         (1)

 

So, if we have two matrices A and B acting on the same vector, we can now see why they should have the same number of rows and columns. They should have the same number of

columns because they both act on the same vector. They should have the same number of

rows because they should each take that vector to the same space.

 

For example, here’s an example of what can go wrong when we try to add two matrices of different sizes.

 

Consider

 

(1 3 2) (3)        (1*3 + 3*2 + 2*1)       (11)

(2 4 1) (2)   =  (2*3 + 4*2 + 1*1)  =  (15)

(4 5 1) (1)        (4*3 + 5*2 + 1*1)       (23)

 

 

Then

 

            (1 2 5)  (3)                   (1 3 2) (3)                    ( 7)         (11)

            (3 4 0)  (2)       +          (2 4 1) (2)        =         (17)    +   (15)

                         (1)                   (4 5 1) (1)                                   (23)

 

And we have trouble, as the two vectors are different sizes. One lives in the 2dimensional plane, one lives in 3space. There is no way we can write down one matrix to represent the action of the two matrices.

 

 

 

[5] MATRIX NOTATION

When proving a mathematical theorem, it is not enough to check it on a couple of matrices. For example:

 

            CLAIM: For any matrix A, A + A is the zero matrix.

 

            FALSE PROOF:

 

                        (0 0)           (0 0)           (0 0)

                        (0 0)    +    (0 0)    =    (0 0)

 

            But ANY other matrix will not work. If you are trying to disprove a claim, it is enough to show that, for a specific example, it fails.

 

            Hence

                        (1 2)          (1 2)           (2 4)

                        (3 4)    +   (3 4)    =     (6 8)

 

 

So it is very useful in mathematics to handle a large number of matrices all at once. We don’t have the time to check each and every matrix individually, as there are infinitely many matrices!

 

So, we develop shorthand notation. We represent an arbitrary entry of a matrix A by

                                                a­i,j

 

The ‘i’ stands for the ith row, the ‘j’ stands for the jth column. So, a12 means the 1st entry in the 2nd row, a22 means the 2nd entry in the 2nd row, and so on.

 

So, we write an arbitrary 2x2 matrix by

 

(a11 a12)

            (a21 a22)

 

We write an aribrary 2x3 matrix by

 

(a11 a12 a13)

            (a21 a22 a23)

 

We write an arbitrary 3x3 matrix by

 

(a11 a12 a13)

            (a21 a22 a23)

            (a31 a32 a33)

 

And we write an arbitrary mxn music (m rows, n columns) by

 

(a11 a12 a13        ...         a1n)

            (a21 a22 a23        ...         a2n)

            (a31 a32 a33        ...         a3n)

            (                       .               )

            (                       .              )

            (                         .             )

            (am1 am2 am3      ...         amn)

 

 

So, to revisit Matrix addition:

 

 

(a11 a12 a13)       +          (b11 b12 b13)      =          (a11+b11   a12+b12   a13+b13)

            (a21 a22 a23)                   (b21 b22 b23)                  (a21+b21   a22+b22   a23+b23)

 

Or, in a specific example:

 

            (1 2 3)              +          (1 0 2)              +          (2 2 5)

            (4 5 6)                          (3 1 0)                          (7 6 6)

 

 

 

 

[6] TRANSPOSE

We now define the transpose of a matrix. For us, the main use will be in studying symmetric matrices, matrices that are equal to their transpose.

 

We write AT for the transpose of the matrix A, and we form AT as follows: the first row of A becomes the first column of AT; the second row of A becomes the second column of AT; the third row of A becomes the third column of AT; ... ; the last row of A becomes the last row of AT.

 

So, if A has 3 rows and 5 columns, then AT has 3 columns and 5 rows (or as we’d normally write it, 5 rows and 3 columns).

 

Let’s do an example:

 

                                    (0 1 1)                                          (0 1)

A         =          (1 2 3)              then  AT     =        (1 2)

                                                                            (1 3)

 

 

Or

                                    (1 2 3 4)                                     (1 0 5)

            A         =          (0 0 1 2)           then AT     =       (2 0 4)

                                    (5 4 3 2)                                     (3 1 3)

                                                                                      (4 2 2)

 

So, for a 2x3 matrix

 

           (a11 a12 a13)                                            (a11 a21)

A   =    (a21 a22 a23)                   then AT    =     (a12 a22)

                                                                                    (a13 a23)

           

[7] SYMMETRIC MATRICES

Symmetric matrices are very useful in mathematics, physics, and engineering. First, the definition. We say a matrix A is symmetric if it equals it’s tranpose, so A = AT. Later we’ll briefly mention why they are useful.

 

The first thing we note is that for a matrix A to be symmetric A must be a square matrix, namely, A must have the same number of rows and columns.  Why? If A has m rows and n columns then AT has n rows and m columns. Since they’re equal, they must have the same number of rows (hence m = n) and the same number of columns (hence n = m). We call matrices with the same number of rows and columns square matrices.

 

For example,

 

            (1 2)        

            (3 4)                

 

even though the above is a square matrix, is not symmetric, as it’s tranpose is

 

            (1 3)

            (2 4)

 

However,

 

            (1 5)

            (5 1)

 

is symmetric, as it does equal its tranpose.

 

THEOREM: Let A a 2x2 matrix. Then A is Symmetric if it’s lower left and upper right entries (a21 and a12) are the same.

 

Proof: We write A as [using a,b,c,d instead of a11, ... as it’s easier to view]

 

            (a b)

            (c d)

 

Then AT  is

 

            (a c)

            (b d)

 

And A = AT means

 

            (a b)                 (a c)

            (c d)     =          (b d)

Since the two matrices are equal, they must be equal componentwise. So the two upper left entries must be the same. This gives a = a, which imposes no new conditions. Let’s look at the other entires. The upper right entires must be the same, which imposes the condition

 

                        b = c.

 

The lower left entries must be the same, which imposes the condition c = b (which we already had), and the two lower right entries must be the same, which imposes d = d.

 

Hence for a 2x2 matrix A to be symmetric we must have b = c, so the matrix looks like

 

            (a b)

            (b c)

 

What about a 3x3 matrix? Assume a 3x3 matrix A equals its transpose:

 

            (a b c)                          (a d g)

            (d e f)               =          (b e h)

            (g h i)                           (c f  i)

 

This gives nine conditions:

 

            a = a

            b = d

            c = g                these come from looking at the first row of each side of the above.

 

            d = b (already had)

            e = e

            f = h                 these come from looking at the second row of each side

 

            g = c (already had)

            h = f  (already had)

            i = i

 

Hence the most general 3x3 symmetric matrix looks like

 

            (a b c)

            (b e f)

            (c f  i)

 

We can, of course, continue to do this for 4x4, 5x5, ..., nxn, ... matrices. The main thing to notice is that symmetric matrices are ‘nice’ with respect to the main diagonal. (Recall the main diagonal is a11, a22, ..., ann. We see that for a symmetric matrix, the entry in the ith row and jth column is the same as the entry in the jth row and ith column).

 

THEOREM: (A + B)T  = AT + BT (or, the transpose of a sum is the sum of the transposes).

 

Proof: Let’s do a specific case first.

 

                        (1 2 3)                          (3 2 1)

            A  =     (4 5 6)              B   =    (2 1 0)

 

 

                        (1 4)                             (3 2)                             (4 6)

Then  AT    =   (2 5)                 BT  =   (2 1)   and  AT + BT =  (4 6)

                        (3 6)                             (1 0)                             (4 6)

 

 

And we find that

 

                             (4 4 4)                                 (4 6)

            A + B  =   (6 6 6)     and (A + B)T  =  (4 6)

                                                                         (4 6)

 

 

Hence we see that (A + B)T = AT + BT for these two matrices!!!

 

Note that the above is NOT a proof – it is merely a verification in this one particular case. Here’s a sketch of the proof.

 

Consider an arbitrary row, say the 2nd. We want to show that (A + B)T = AT + BT are the same. We’ll do this by showing that each column on the left hand side equals the corresponding column on the right hand side.

 

Let’s look at the LHSide first. We add the 2nd row of A to the 2nd row of B, and then this sum becomes the 2nd row of A + B. Taking transposes, this gives the 2nd row of (A + B)T.

 

Now we examine the RHSide. The 2nd column of AT is the 2nd row of A; the 2nd column of BT  is the 2nd row of B. So the 2nd column of  AT + BT is the 2nd row of A plus the 2nd row of B.

 

So, the 2nd row of (A + B)T equals the 2nd row of AT + BT. But there is nothing special about 2 – we could do this equally well for any column, and we see the two sides are in fact equal.

 

As promised, a few words about why symmetric matrices are useful. First, they’re easier to handle then general matrices, as they only need about half as many entries. Once you specify the entries on the main diagonal and above the diagonal, you know all the entries (as the entries below the diagonal equal the ones above the diagonal). You’ve seen in your engineering course one example of where symmetric matrices arise. One common example in mathematical physics is with the matrix of second derivatives. For example, consider the matrix where

                        a­i,j  =  df/dxidxj.

 

Here f is a function of n variables (x1, ..., xn), and df/dxidxj is the partial derivative of f with respect to xi and xj. For “good” functions f we have df/dxidxj  =  df/dxjdxi (or, it doesn’t matter which order you take the derivatives).

 

 

 

 

[8] BASIC FACTS ABOUT MATRICES

 

[1] A + B = B + A

[2] x(A + B) = xA + xB,           where x is any number

[3] (x+y)A = xA + yB

[4] AB does not always equal BA

[5] A(BC) = (AB)C

[6] A(BC) does not always equal (AC)B (for example, consider A = I)

[7] AA-1 = I, the Identity matrix

[8] (AT)T = A

[9] (A + B)T  = AT + BT

[10] (xA)T = x AT

[11] (AB)-1 = B-1 A-1

[12] (AB)T = BT AT

[13] (A-1)T = (AT)-1

 

Note: we define, for x a real number and A a matrix, xA to be the matrix whose entries are x times those of A.

 

Example:

 

               (1 2)      (2 4)

            2 (0 1)  =  (0 2)

               (3 4)      (6 8)