NOTES ON LINEAR ALGEBRA

 

CONTENTS:

[13] DOT PRODUCTS

[14] DETERMINANTS - I

 

 

[13] DOT PRODUCTS

 

The Dot Product is a function from pairs of vectors to numbers. So, the input is two vectors, say v = (x1, y1) and w = (x2, y2). We use a dot, ·, to represent the Dot Product.

 

Let |v| denote the length of the vector v.

 

                                                v  =  (x1, y1)

                                                 

 

 

                                                                this vector has length y1      

 

 

                  

                             this vector has length x1

 

 

 

So by the Pythagorean Theorem, the vector v  has length Sqrt[x12 + y12].

 

Hence |v| = Ö x12 + y12. Similarly |w| = Ö x22 + y22. We now define the Dot Product:

 

          v · w = x1 x2 + y1 y2

 

We will show later that the Dot Product has a very special property, which will explain its usefulness. Namely, if we have

 

 

 

                                w

 

 

 

 

 

 


                                                                                v

                        q

 

 

 

 

So, if q is the angle between v and w, then it is a theorem that

 

          v  · w  =  |v| |w| cosq

 

 

As always, we will only prove this in two dimensions. Let’s look at some special cases. Consider two vectors v and w that are perpendicular, for example:

 

                        w = (0,y2)

 

 

 

 

 

 

 


                                                                                          v = (x1, 0)

 

 

Then v  · w = x1 0 + 0 y2 = 0. But as q = 90, cosq = 0, so the formula holds in this case!

 

Now let’s consider v and w in the same direction, say along the x-axis:

 

 

                                v = (x1,0)              w = (x2, 0)

 

 

 


Then v  · w = x1 x2. But here q = 0, so cosq = 1, and again the formula works.

 

Let’s do a more exotic example. Let’s do v and w in the same direction, but not necessarily along the x-axis.

 

 

 


                                                                             w = (3x,3y)

 

 

 

 

 


                                                    v = (x,y)

                                    

 

 

 

 

Now, |v| = Sqrt[x2 + y2], |w| = Sqrt[9x2 + 9y2], q = 0 so cosq = 1.

 

Then |v| |w| cosq    = Sqrt[x2 + y2] * Sqrt[9x2 + 9y2]

                             = Sqrt[x2 + y2] * 3 Sqrt[x2 + y2]

                             = 3 (x2 + y2)

 

And   v  · w          =       x 3x + y 3y = 3 (x2 + y2).

 

So again, the formula is true.

 

We now need a linearity property of the Dot Product. Let’s say we have three vectors u, v, and w. Then

 

          u · (v + w) = u · v  +  u · w

 

The proof is by straightforward computation. Let’s take as our three vectors

 

          u = (x1, y1)

          v = (x2, y2)  

w = (x3, y3)

 

Then v + w = (x2 + x3, y2 + y3)

and u · (v + w)      =  x1  (x2 + x3) + y1 (y2 + y3)

                             =  x1 x2  + x1 x3 + y1 y2 + y1 y3

                                                = x1 x2  + y1 y2 + x1 x3 + y1 y3

                             = u · v  +  u · w

 

 

Using all the junk we’ve just proved, we can now show

v  · w  =  |v| |w| cosq

 

Consider two vectors v = (a,b) and w = (c,d):

 

                                                w = (c,d)

 

 

 

 

 

 


                                        q                                                                v = (a,b)

 

 

 

 

We break w up into two different vectors:

          wperp, which is perpendicular to v, and wpara, which is parallel to v.

 

By the above, we have

 

          v · w  = v · (wperp + wpara) = v · wperp + v · wpara

 

But v · wperp = 0, and by the special case, v · wpara = |v| |wpara| cosqv wpara

 

where  qv wpara is the angle between v and wpara. But this angle is 0, so we get

 

v · w = v · wpara = |v| |wpara­|

 

However, we know what |wpara| is – it’s just |w| cosq. Why? wpara is the base of a right triangle with hypotenuse w and angle q. So substituting above for |wpara| yields

 

          v · w  =  |v| |w| cosq

 

So we have proved the result in two dimensions. If we were working in 3 space, where we’d have vectors like

 

          v = (x1, y1, z1)

          w = (x2, y2, z2)

 

then analogously we define v · w = x1 x2 + y1 y2 + z1 z2. Since any two vectors lie in a plane (doesn’t matter how many dimensions we are in) we can still talk about the angle between two vectors, and the analogous statement is true.

 

The three things to take away from Dot Products are:

 

          [1] Two vectors have dot product zero if and only if they are perpendicular

          [2] The dot product of two vectors is the product of their lengths if and only if the two vectors are parallel.

          [3] The Dot Product measures the angle between two vectors. More precisely,  cosq =  v · w / |v| |w|. So, if I know the length of two vectors AND if I know their dot product, I can immediately measure the angle between them!

 

 

 

 

 

 

[14] DETERMINANTS - I

 

There are several interpretations for Determinant. For now, we will view it as a function whose input is a SQUARE matrix and whose output is a number. We will see in the 2x2 case that this number is the AREA of the parallelogram formed by the rows of A.

 

If the rows of A are parallel, then this parallelogram will have zero area; if the rows of A aren’t parallel, then this parallelogram will have non-zero area. So, the Determinant provides a quick check of whether or not two vectors are in the same direction.

 

In the plane, this isn’t too important; however, in higher dimensions it becomes indispensible. Let’s say we are in 3-dimensional space. A is a 3x3 matrix, so its three rows give us three vectors. They form the generalization of a parallelogram, a parallelpiped. (I may have the terminology wrong – it’s been a long time since I’ve used these words!). So instead of talking about area, we should talk about volume. If the three vectors lie in one plane, then this parallelpiped will have zero volume. If the three vectors don’t lie in one plane, then the parallelpiped will have non-zero volume. So for 3x3 matrices, the Determinant will measure whether or not the three rows lie in a plane, or if they ‘fill’ all of three space. Eventually we’ll see this is related to questions of when is a matrix invertible.

 

Now for the definition for 2x2 matrices.

 

                             (a b)

Let     A       =       (c d)

 

Then we denote Determinant(A) several ways:

 

                                                |a b|

Determinant(A) = Det(A)   =      |c d|  = ad - bc

 

Let’s see that Det(A) does give the area in certain special cases.

 

 

 

 

CASE 1:

          (  a   b)

A  =   (3a 3b)

 

 

Then Det(A) = a 3b - b 3a = 0.

 

Note there’s nothing special about 3:

 

          (  a    b)

A   = (ma mb)

 

Then Det(A) = a mb - b ma = 0.

 

So, when one row is parallel to another, we do get Det(A) = 0!

 

 

CASE 2:

 

          (a 0)

A   =           (c d)

 

                                  (c,d)

 

 

 

 

                                                                                                                                                                                                (a,0)

 

 

The base of the parallelogram is a, the height is d. Hence the area is ad.

But Det(A) = ad - 0c = ad. So in this case, it works.

 

 

Now we consider the general 2x2 case and, using the Dot Product, we’ll prove that Det(A) = area of parallelogram formed by the rows of A.

 

 

                             (a b)

Again, take A   =   (c d)

 

                               w =  (c,d)

 

 

 

                                                            wperp

 

 

 

                                                                                                               v =(a,b)

 

 

 

 

 


                          q    wpara

 

 

 

 

 

|v|2 = a2 + b2 and |w| = c2 + d2 (by the pythagorean theorem).

 

|wpara| = |w| cosq, |wperp| = |w| sinq.

 

So the area of the parallelogram is |v| |wperp|, or

 

          Area = |v| |wperp| = |v| |w| sinq

 

But cos2q + sin2q = 1, so sinq = Sqrt[1 - cos2q].

 

Moreover, |v| |w| cosq = v · w = ac + bd.

 

Dividing by |v| |w| yields  cosq = (ac+bd) / |v| |w|

 

Hence Area = |v| |w| Sqrt[1 - cos2q]

                   = |v| |w| Sqrt[1 - (ac+bd)2 / |v|2 |w|2 ]

                   = Sqrt[|v|2 |w|2 - (ac+bd)2]

 

Substituting for |v|2 = a2 + b2 and |w| = c2 + d2 yields

 

          Area   = Sqrt[ (a2 + b2)( c2 + d2) - (ac+bd)2 ]

                   = Sqrt[ a2c2 + a2d2 + b2c2 + b2d2 - a2c2 - 2acbd - b2d2]

                   = Sqrt[a2d2 + b2c2  - 2acbd ]

                   = Sqrt[a2d2 - 2adbc + b2c2]

                   = Sqrt[ (ad - bc)2 ]

                   = ad - bc

 

So the area of the parallelogram is ad - bc, which is just Det(A)!

 

One of the reasons Determinant is such a useful function is that say we start with a matrix A, and we do Gaussian Elimination, ending up with a matrix B. Then A and B have the same determinant!

 

The reason is Gaussian Elimination is just adding multiples of one row to another. So, let’s start with the matrix

 

                             (a b)

          A       =       (c d)

 

 

 

 

               w = (c,d)

 

 

 

 

                                                                                      v = (a,b)

 

(To simplify things, I’m drawing it as if v is along the x-axis, though the method of proof works in general. This just makes the pictures look nicer).                                                                                                                                 

Now let’s say we add on a small multiple of (a,b) to (c,d). So we have a new vector w’ = (c+ma, d+mb). Geometrically:

 

               w = (c,d)

                                          w’

                             

 

 

                                                                                      v = (a,b)

 

 

Notice that they have the same base, and the same size height! Hence the two areas are the same.

 

We can also argue algebraically:

 

 

Det(A) = ad - bc.

 

          (   a           b   )

B  =   (c+ma  d+mb)

 

Then Det(B) = a(d+mb) - b(c+ma)

                    = ad + mab - bc - mab = ad - bc