-
Mathematica code snippet for the week: Here is a nice way to program a
variable number of for loops. This is great when you initially do not know how
many For loops you'll need. The trick is to use a base B expansion.
-
varforloop[n_, B_] := Module[{},
For[m = 0, m <= B^n - 1, m++,
{
list = IntegerDigits[m, B, n];
(* elements of list are the values of x_1, ..., x_n *)
}];
];
-
varforloop[n_, B_] := Module[{},
- Review: The cumulative
distribution function
- The cumulative distribution function is one of the key tools of the subject, and gives a sense of why continuous random variables are easier to analyze than discrete; namely, for continuous we have the Fundamental Theorem of Calculus at our disposal to pass from a cumulative distribution function to a density; we do not have differentiation available in the discrete case. Note that a cumulative distribution function does not determine a unique density; however, it almost does so, as any two densities must integrate to the same value on any interval. (The technical jargon is to say that the density is determined up to a function which is zero almost everywhere.) If there is interest, let me know and I'll talk a bit about the basics of measure theory (and show that almost no numbers are rational in the sense of measure).
- We began today with a quick review of
Taylor series. We
talked a lot about the consequences of a specific function's Taylor series.
-
The first step in any investigation is to figure out what questions to ask. Here are the two standard ones: (1) does the Taylor series exist (or for what x does it converge and equal the original function), and (2) is the Taylor series unique? The answers were surprising; a Taylor series must converge at the expansion point, but it's possible to only converge there; it's also possible for two different, infinitely differentiable functions to have the same Taylor series!
-
Analysis is hard. The function f(x) = exp(-1/x2) if x is not zero and 0 otherwise has all of its derivatives vanish at 0, but its Taylor series agrees with the original function only at x=0 (which is nothing to be proud of!). Complex analysis is quite different; there if a function is complex differentiable once then it is infinitely complex differentiable, and it equals its Taylor series in a neighborhood of the point. This fact is one reason why we frequently use characteristic functions instead of generating or moment generating functions (which we'll cover later in the semester). We also discussed the similarities between how Taylor coefficients uniquely determine a nice function and how moments uniquely determine a nice probability distribution. It is sadly not the case that a sequence of moments uniquely determines a probability distribution; fortunately in many applications some additional conditions will hold for our function which will ensure uniqueness. For the non-uniqueness of Taylor series, the standard example to use is f(x) = exp(-1/x^2) if x is not zero and 0 otherwise. To compute the derivatives at 0 we use the definition of the derivative and L'Hopital's rule. We find all the derivatives are zero at zero; however, our function is only zero at zero. We will see analogues of this example when we study the proof of the Central Limit Theorem.
-
Here's a fun application of Taylor series: we can prove all trig identities!
- Using the
Taylor series expansions for cosine
and sine, we find e^(iθ) = cos θ + i sin θ. From this we find |e^(iθ)| =
1; in fact, we can use these ideas to prove all trigonometric identities!
For example:
- Inputs: e^(iθ) = cos θ + i sin θ and e^(iθ) e^(iφ) = e^(i (θ+φ))
- Identity: from e^(iθ) e^(iφ) = e^(i (θ+φ))
we get, upon substituting in the first identity, that (cos θ + i sin θ) (cos
φ + i sin φ) = cos(θ+φ) + i sin(θ+φ). Expanding the left hand side gives (cos
θ cos φ - sin θ sin φ) + i (sin θ cos φ + cos θ sin φ) = cos(θ+φ) + i
sin(θ+φ). Equating the real parts and the imaginary parts gives the
identities
- cos(θ+φ) = cos θ cos φ - sin θ sin φ
- sin(θ+φ) = sin θ cos φ + cos θ sin φ
- Of course, we have to be careful. Note
above we are using \(e^x e^y = e^{x+y}\); if this were false it would be
bad notation, but we
must make sure it is true before using. This inequality is a
statement about three infinite series, namely the product of the first two
is the third. We must show \(\sum_{n=0}^\infty x^n/! \sum_{m=0}^\infty
y^m/m!\) equals \(\sum_{\ell=0}^\infty (x+y)^\ell/\ell!\).
- The double sum converges absolutely, so we can re-order the summands (this is essentially Fubini's theorem, which you might've seen in Calc III or Real Analysis).
- We collect all products whose power sums to \(\ell\) and find the double sum is \(\sum_{\ell=0}^\infty \sum_{k=0}^\ell x^k y^{\ell-k} / k! (\ell-k)!\).
- A large part of mathematics is pattern recognition. Notice what we have is close to \(\left({\ell \atop k}\right)\), the binomial coefficient. We thus multiply by 1 in the form \(\ell!/\ell!\), notice the binomial coefficient, apply the binomial theorem and end with the claimed result.
- Using the
Taylor series expansions for cosine
and sine, we find e^(iθ) = cos θ + i sin θ. From this we find |e^(iθ)| =
1; in fact, we can use these ideas to prove all trigonometric identities!
For example:
-
- Onion trig article: http://www.theonion.com/articles/nations-math-teachers-introduce-27-new-trig-functi,33804/
- For real trig functions: http://www.3quarksdaily.com/3quarksdaily/2013/09/10-secret-trig-functions-your-math-teachers-never-taught-you.html
- Here are a lot
of comments on Taylor series from when I taught Math 105 (now Math 150) a few years back.
- We saw how well Taylor series approximate functions. A Mathematica program here is (hopefully) easy to use. You can specify the point and number of terms of the Taylor series of cos(x) to do. At first it might seem surprising that there is no improvement in fit when we go from a second order to a third order Taylor series approximation; however, we have cos(x) = 1 - x^2/2! + x^4/4! - x^6/6! + .... In other words, all the odd derivatives vanish at the origin, and thus there is no improvement at the origin in adding a cubic term (ie, the best cubic coefficient at the origin is 0). If we go to a fourth order, we do see improvement. By n=10 or 12 we are already getting essentially an entire period correct; by n=40 we have several cycles.
- Taylor's theorem one of the most important applications of calculus. It
allows us to replace complicated functions with simpler ones. There are
numerous questions to ask.
- Are Taylor series unique? Yes. The definition just involves taking sums of derivatives; the process is well-defined.
- Does every infinitely differentiable function equal its Taylor series expansion? Sadly, no; the function f(x) = exp(1/x^2) if |x| > 0 and 0 if x=0 is the standard example. This function causes enormous problems in probability. There are many functions which do equal their own Taylor series expansion, such as exp(x), cos(x) and sin(x). It's not surprising that these three are listed together, as we have the wonderful Euler-Cotes formula: exp(i x) = cos(x) + i sin(x), with i = sqrt(-1). At first this formula doesn't seem that important; after all, we mostly care about real quantities, so why complexify our life by adding complex (i.e., imaginary) numbers? Amazingly, even for real applications (applications where everything is real), complex numbers play a pivotal role. For example, note that a little algebra gives cos(x) = (exp(i x) + exp(-i x)) / 2 and sin(x) = (exp(i x) - exp(-i x)) / 2i. Thus properties of the exponential function transfer to our trig functions. The hyperbolic cosine and sine functions are similarly defined; cosh(x) = cos(i x) = (exp(-i x) + exp(x)) / 2. The Foxtrot strip below (many thanks to the author, Bill Amend, for permission to post) illustrates the confusions that can happen between hyperbolic and regular trig functions (for extra credit, why does Eugene know that the calculator cannot be giving the right answer?). It's worth noting that the formula exp(i x) = cos(x) + i sin(x) allows us to derive ALL trig identities painlessly! See the comments immediately above.
-
- FoxTrot (c) Bill Amend. Used by permission of Universal Uclick. All rights reserved.
- How easy are Taylor series to use? If we keep just a few terms, it's not too bad; however, as the great Foxtrot strip below shows, it's not always clear how nicely something simplifies.
-
- FoxTrot (c) Bill Amend. Used by permission of Universal Uclick. All rights reserved.
- In the strip above, notice the large factorials in the denominator. Note 52! is about 1068; in other words, these terms are small! For interest, 52! is the number of ways (with order mattering) of arranging a standard deck of cards. There are about 1085 or so subatomic thingamabobs in the universe; we see quite quickly reach numbers this high (a deck with 70 cards more than sufices; in other words, we could not have each subatomic object in the universe represent a different shuffling of a deck of 70 cards). In a related note, it's important to think a bit and decide what 0! should be. It simplifies many formulas to have 0! = 1, and we can make this somewhat natural by saying there is only 1 way to do nothing (mathematically, of course). The definition of the factorial function on Wikipedia talks a little bit about this.
- Unlike 0!, 0^0 is a bit more controversial as to what the definition should be. As I don't want to pressure anyone, I will not publically disclose where I stand in the great debate, though I'm happy to tell you privately / through email.
- It's worth remarking on why we have n! in the denominators. This is to ensure that the nth derivative of our function equals the nth derivative of the Taylor expansion at the point we're expanding. In other words, we're matching up to the first 2 derivatives for the 2nd order Taylor expansion, up to the first 3 for the 3rd order Taylor expansion, and so on. It isn't surprising that we should be able to do a good job; the more derivatives we use, the more information we have on how the function is changing near the key point.
- For many purposes, we just need a first order or second order Taylor series; one of my favorites is the proof of the Central Limit Theorem in probability. One of my favorite proofs involves second order Taylor expansions of the Fourier Transforms (these were mentioned in the additional comments on Friday, March 12).
- If f(x) equals its infinite Taylor series expansion, can we differentiate term by term? This needs to be proved, and is generally done in a real analysis course. For some functions such as exp(x) we can justify the term by term differentiation, but note that this is something which must be justified.
- One of the most important applications of
finding areas under curves is in probability, where we may interpret these
areas as the probability that certain events happen. Key concepts are:
- Mean or Expected Value
- Standard Deviation
- Skewness and kurtosis (for the hypercompetitive students who really want to compare themselves to the class).
- We talked a lot about the Cauchy distribution, and how it does not have a finite mean or variance. It does play a major role in some economics theories, discussed in the next comments. The starting point of these discussions is linearity of expectation; while we didn't prove it today, we saw it was reasonable. We'll see that variances cannot be linear just by looking at X+X and X-X.
-
Economics: the standard random
walk hypothesis seems to have
lost most of its supporters, though there are variants (and I'm not familiar
with all); see also the efficient
market hypothesis and technical
analysis, and all the links there. (There are also many good links on the
wikipedia page on Eugene
Fama). Two famous books (with different conclusions) are Malkiel's A
random walk down wall street and
Mandelbrot-Hudson's The
(mis)behavior of markets (a fractal view of risk, ruin and reward). Some
interesting papers if you want to read more:
- Mandelbrot: Variation on certain speculative prices (a must read!)
- Fama: Mandelbrot and Stable Paretian Hypothesis
- Fama: Random Walks Stock Prices
- For more on randomness, check out The Black Swan by Taleb (amazon.com page here, wikipedia page here). Several members of the class have recommended this book highly, and from reading excerpts on the web I understand why.
- For more on fractal geometry, click here. We did the Koch snowflake; another popular one is the Cantor set. See here for fractal dimensions. To actually compute pictures of items like the Mandelbrot set, one needs to iterate polynomials. This can lead to the fascinating subject of efficient algorithms; when I wrote such programs years ago on what would now be considered `slow' computer, I had to use Horner's algorithm to get things to run in a reasonable time.
-
We recalled a powerful technique
from Calc I: if f(g(x)) = x (so f and g are inverse functions, such as sqrt(x^2)
or, the one we needed today for the Cauchy distribution, tan(arctan(x))), then
g'(x) = 1 / f'(g(x)); in other words, knowing the derivative of f we know the
derivative of its inverse function.
As the chain rule and inverses are so useful, let's go through the theory.
- If \(f(g(x)) = x\) (so \(f\) and \(g\) are inverse functions, such as \(\sqrt{x^2})\), then using the Chain Rule to take derivatives gives \(g'(x) = 1 / f'(g(x))\); in other words, knowing the derivative of \(f\) we know the derivative of its inverse function \(g\). This was used in Calc I to pass from knowing the derivative of \(\exp(x)\) to the derivative of \(\ln(x)\). We can apply this to various inverse trig functions (a list of the derivatives of these are here). This highlights one of the most painful parts of integration theory -- just because we are close to finding an anti-derivative does not mean we can actually find it! While there is a nice anti-derivative of \(\sqrt{1 - x^2}\), it is not a pure derivative of an inverse trig function. There are many tables of anti-derivatives (or integrals) (a fun example on that page is the Sophomore's Dream). Unfortunately it is not always apparent how to find these anti-derivatives, though of course if you are given one you can check by differentiating (though sometimes you have to do some non-trivial algebra to see that they match). In fact, there are some tables of integrals of important but hard functions where most practitioners have no idea how these results are computed (and occasionally there are errors!). We will see later how much simpler these problems become if we change variables; to me, this is one of the most important lessons you can take from the course: Many problems have a natural point of view where the algebra is simpler, and it is worth the time to try to find that point of view!
- Let \(f(x) = \exp(x)\). Then \(f'(x) = \lim [f(x+h) - f(x)]/h\) = \(\lim_{h \to 0} [\exp(x+h) - \exp(x)] / h\) = \(\lim_{h \to 0} [\exp(x) \exp(h) - \exp(x)] / h\) = \(\exp(x) \lim_{h \to 0} [\exp(h) - 1] / h\). As \(\exp(0) = 1\), we find \(f'(x) = \exp(x) \lim_{h \to 0} [f(h) - f(0)] / h\) = \(\exp(x) f'(0)\); thus we know the derivative of the exponential function everywhere once we know the derivative at 0!