Honors 1: Undergraduate Math Lab^{∗}

Peter Sarnak^{†}, Steven Miller^{‡}, Alex Barnett^{§}

September 11th, 2002

Courant Institute of Mathematical Sciences New York University New York, NY

Abstract

Quotient groups. Basic probability theory for random walk. Lecture by Steven Miller; notes by Alex Barnett and Steven Miller.

1 Quotient groups

Say we have a finite Abelian group G (this means for all x, y ∈ G, xy = yx) of order m which has a subgroup H of order r. We will use multiplication as our group operation. Recall the coset of an element g ∈ G is defined as the set of elements gH = g{h_{1}, h_{2}, ⋯, h_{r}}. Since G is Abelian (commutative) then gH = Hg and we will make no distinction between left and right cosets here.

The quotient group (or factor group), symbolized by G/H, is the group formed from the cosets of all elements g ∈ G. We treat each coset g_{i}H as an element, and define the multiplication operation as usual as g_{i}Hg_{j}H. Why do we need G to be Abelian? The reason is we can then analyze g_{i}Hg_{j}H, seeing that it equals g_{i}g_{j}HH. We will analyze this further when we prove that the set of cosets is a group.

∗Homepage: http://www.math.nyu.edu/∼millerj/

†E-mail: sarnak@math.princeton.edu

‡E-mail: millerj@cims.nyu.edu or sjmiller@math.princeton.edu

§E-mail: barnett@nmr.mgh.harvard.edu

There are several important facts to note. First, if G is not Abelian, then the set of cosets might not be a group. Second, recall we proved the coset decomposition rule: given a finite group G (with n elements) and a subgroup H (with r elements) then there exist elements g_{1} through g_{k} such that

G = ⋃_{i=1}^{k} g_{i}H.  (1)

The choices for the g_{i}’s is clearly not unique. If g_{1} through g_{k} work, so do g_{1}h_{1} through g_{k}h_{k}, where h_{i} is any element of H. Recall this was proved by showing any two cosets are either distinct or identical.

We will show below that, for G Abelian, the set of cosets is a group. Note, however, that while it might at first appear that there are many different ways to write the coset group, they really are the same. For example, the cosets gH and gh_{1}h_{2}^{4}h_{3}H are equal. This is similar to looking at integers mod n; mod 12, the integers 5, −7 and 19 are all equal, even though they look different.

We now prove that the set of cosets is a group (for G Abelian).

Closure. By commutivity g_{i}Hg_{j}H = g_{i}g_{j}HH. What is “HH”? Just the set of all r^{2} possible combinations of elements of H. By closure, and the existence of the identity, this just gives H again (recall no element in a group can appear more than once—duplicates are removed). Therefore g_{i}Hg_{j}H = g_{i}g_{j}H. Now, as G is a group and is closed, g_{i}g_{j} ∈ G. Thus, there is a α such that g_{i}g_{j} ∈ g_{α}H (as G = ⋃_{β=1}^{k} g_{β}H. Therefore, there is an h ∈ H such that g_{i}g_{j} = g_{α}h, which implies g_{i}g_{j}H = g_{α}hH = g_{α}H. Thus, the set of cosets is closed under coset multiplication. Note, however, that while the coset g_{i}g_{j}H is in our set of cosets, it may be written differently.

Identity. If e is identity of G, then eHg_{i}H = g_{i}H and g_{i}HeH = g_{i}H, so eH is the identity of this quotient group.

Associativity. Since as you may have noticed, the quotient group elements behave just like those of G, associativity follows from that of G.

Inverse. It is easy to guess g^{−1}H is the inverse of gH. Check it: g^{−1}HgH = g^{−1}gH = eH = identity, also true the other way round of course by commutativity. Unfortunately, g^{−1}H might not be listed as one of our cosets! Thus, we must be a little more careful. Fortunately, as g^{−1} ∈ G = ⋃_{β=1}^{k} g_{β}H, there is an α such that g^{−1} ∈ g_{α}H. Then, there is an h ∈ H with g^{−1} = g_{α}h. Thus, g^{−1} = g_{α}hH = g_{α}H, and direct calculation will show that the coset g_{α}H is the inverse (under coset multiplication) of gH.

2 Random walk and discrete probability

Each step in a random walk is a random event. We first study a single random event, then the combination of two random events, then multiply repeated events.

For other introductory probability theory see

http://engineering.uow.edu.au/Courses/Stats/File24.html

2.1 Probability distribution for a single event, mean, variance

Our single event consists of one of a set of choices happening. The choices are labelled by i = 1 ⋯ N, which are exclusive (no more than one can happen), and complete (no less than one can happen). For instance, for a single coin toss,

i = 1  H, heads,
i = 2  T, tails.

We take the choice i as a random variable, meaning all we know is a probability p_{i} ≥ 0 that each choice can happen. p = 0 means it never happens, p = 1 means it always happens, and most things are somewhere in between (the unbiased coin has p_{i} = \frac{1}{2}, ∀i, ignoring the small probabilities of the coin landing on its edge or quantum tunneling through the table.)

The set of {p_{i}} we call the probability distribution. Completeness implies

∑_{i} p_{i} = 1.  (2)

Note the abbreviation ∑_{i} for ∑_{i=1}^{N}.

We have some quantity f which has a value f_{i} for each choice i. For instance, f could be the number of dots on each face i of a die, in which case f_{i} = i. Mean and variance are ways to characterize (summarize) the distribution over f.

Mean. The mean or expectation value (expected value) of f over the distribution is

f̄ ≡ E[f] = ∑_{i} p_{i}f_{i}.  (3)

This is just a weighted average of f. Check it for the (unweighted) dice. p_{i} = \frac{1}{6}, ∀i. You should get f̄ = \frac{7}{2}.

Variance. The variance is the square of the standard deviation σ_{f}:

σ_{f}^{2} ≡ var[f] = ∑_{i} p_{i}(f_{i} − f̄)^{2}.  (4)

Crudely σ_{f} gives the width of the distribution in f. From the definition you can see σ_{f} is the root mean square (rms) deviation from the mean. Expanding out the square, and using earlier results,

var[f] = ∑_{i} p_{i}f_{i}^{2} − 2f̄· ∑_{i} p_{i}f_{i} + f̄^{2} · ∑_{i} p_{i}
= ∑_{i} p_{i}f_{i}^{2} − f̄^{2}
= E[f^{2}] − E[f]^{2}.  (5)

This is a very useful formula. The first term is often known as the second moment of f. The first moment is just the mean. The m^{th} moment is defined as ∑_{i} p_{i}f_{i}^{m}.

A quick comment about units. If f_{i} and f are measured in feet (for example), the variance (which gives information on how spread out f_{i} is) has units feet-squared. If someone asks how tall the people in our class are, one would answer about 5\frac{1}{2} feet. If one is further pressed to give a range for the heights of our class, one might say 5\frac{1}{2} feet, plus or minus \frac{1}{4} of a foot. One would not give the error range in feet-squared! Thus, in measuring error it is the square-root of the variance that comes into play. Note that if f_{i} is in feet, the variance is in feet-squared, and the square-root of the variance is in feet.

2.2 Multiple events

Consider two random events. Let the first have choices i = 1 ⋯ N (with probabilities p_{1} through p_{N}); let the second event have choices j = 1 ⋯ M (with probabilities q_{1} through q_{M}). Then there are NM choices (possibilities) for the combined event, which we could label by k ≡ ij. Since k is also a random variable, it has probability r_{k} = r_{ij} (you could think of this as a matrix in i, j). If the two events are independent (also called uncorrelated), then

r_{ij} = p_{i}q_{j}, ∀ij  independence.  (6)

(In other words, the matrix is separable in the i and j directions). Many events we study will be independent. For example, if you flip a fair coin twice, the result of the first flip has no effect on the result of the second flip. Or if you roll a fair die, et cetera.

As before we have a quantity f_{i} associated with each choice i for the first event. For the second event we have a (in general different) quantity g_{j} for its choice j. We want to learn about the sum of these two quantities,

s ≡ f + g.  (7)

Note that for each combined choice ij, this quantity s has value s_{ij} = f_{i} +g_{j}. How is s is distributed? We will show that independence of the two events implies a simple law giving the mean and variance of s in term of those of f and g.

We compute the mean of s, following Eq. 3, except now we are summing over all combined possibilities ij,

E[s] = ∑_{ij} r_{ij}s_{ij} = ∑_{ij} p_{i}q_{j}(f_{i} + g_{j}) = ∑_{i} p_{i}f_{i} · ∑_{j} q_{j} + ∑_{i} p_{i} · ∑_{j} q_{j}g_{j}
= f̄· 1 + 1 · ḡ = f̄+ ḡ.  (8)

So, the means add.

We isolate this important fact:

Lemma 2.1 For independent events, the mean of a sum is the sum of the means. Equivalently, the sum of the expected values is the expected value of the sums. Thus, for any independent events A and B, E[A + B] = E[A] + E[B].

What if we multiply A by a constant c? For example, consider outcomes A_{i} with probabilities p_{i}. The mean Ā = E[A]. What is the mean of the new event with outcomes cA_{i} occurring with probabilities p_{i}?

Well,

E[cA] = ∑_{i} p_{i}cA_{i}
= c ∑_{i} p_{i}A_{i}
= cE[A].  (9)

We have therefore shown Lemma 2.2 The mean of a multiple is the multiple of the mean. Equivalently, the expected value of a multiple is the multiple of the expected value. Thus, E[cA] = cE[A].

We now calculate the variance of a sum, using the above results. For the variance of s = f + g we can use Eq. 5 to get

var[s] = E[s^{2}] − E[s]^{2}
= E[(f + g)^{2}] − (E[f + g])^{2}
= E[(f + g)^{2}] − (E[f] + E[g])^{2} by Lemma 2.1
= E[(f + g)^{2}] − (f̄+ ḡ)^{2}
= E[f^{2}] + 2E[fg] + E[g^{2}] − (f̄+ ḡ)^{2}.  (10)

We justify the last step as follows:

E[(f + g)^{2}] = E[f^{2} + 2fg + g^{2}]
= E[(f^{2} + 2fg) + g^{2}]
= E[(f^{2} + 2fg)] + E[g^{2}] by Lemma 2.1
= E[f^{2}] + E[2fg] + E[g^{2}] by Lemma 2.1
= E[f^{2}] + 2E[fg] + E[g^{2}] by Lemma 2.2.  (11)

Using independence again we can factorize E[fg] = ∑_{ij} r_{ij}f_{i}g_{j} = ∑_{i} p_{i}f_{i} · ∑_{j} q_{j}g_{j} = E[f]E[g] = f̄ḡ. Elegantly, this term is responsible for cancelling the cross-term in (f̄+ ḡ)^{2}, and collecting the remaining terms leaves

var[s] = var[f] + var[g]  (12)

So, the variances also add.

We now take the special case when the second event is identical to the first. That is, M = N, q_{i} = p_{i}, g_{i} = f_{i}, ∀i. In this case the above shows that s̄ = 2f̄ and var[s] = 2var[f].

We can repeat the above combination law for successively repeated events. Such events are called identical independently distributed (iid) events. Suppose we have K repetitions of iid events, with total s ≡ f + g + ⋯ + z, we can just repeatedly apply the above rules to get

s̄ = K f̄  (13)
var[s] = K var[f].  (14)

The mean and variance are not the only characteristics of the distribution that add like in this way. Amazingly, there is an infinite sequence of special combinations of the higher moments, called cumulants, which add just like this. The mean and variance are just the first two cumulants.

2.3 Simplest random walk

We now have the tools to characterize a random walk. We choose N = 2 and p_{1} = p_{2} = \frac{1}{2}, just as with the coin toss, and define the “step” displacements f_{1} = +1 and f_{2} = −1. This corresponds to a drunkard taking (uncorrelated) steps of unit length along the integer line.

We use Eqs. 3 and 5 to evaluate the mean and variance of a single step event.

f̄ = +\frac{1}{2} − \frac{1}{2} = 0  (15)
var[f] = \frac{1}{2}.(1)^{2} + \frac{1}{2}.(−1)^{2} − (0)^{2} = 1.  (16)

For K steps, starting from the origin, we have the final displacement of s, the sum of all the steps, using the formulae above,

s̄ = K · 0 = 0  (17)
var[s] = K · 1 = K.  (18)

So the standard deviation, i.e. the width of the distribution of s, has value

σ_{s} ≡ \sqrt{var[s]} = \sqrt{K}.  (19)

This is our first vital fact about all but the most pathological^{1} random walks: the distribution has width which scales like K^{1/2}. This means that a typical distance from the origin is \sqrt{K}. This is called a diffusion process and is very common in the real world.

Again, remember that if the person walks in feet, the variance (which is a measure of how much the distribution spreads out) will be in feet-squared. By taking the square-root we again have units of feet.

1Random walks which do not exhibit this power law have infinite variance and exhibit anomalous diffusion. For more on this, see M. Bazant’s excellent course at http://www-math.mit.edu/∼bazant/teach/18.325

2.4 Central Limit Theorem

The Central Limit Theorem (CLT) states that the distribution on s tends to a Gaussian distribution,

p(s) ≈ N (μ, σ^{2}) ≡ \frac{1}{\sqrt{2π}σ} e^{− \frac{(s−μ)^{2}}{2σ^{2}}}  (20)

with mean μ = s̄ and variance σ^{2} = var[s] as given above, as K → ∞. This is a very common “bell curve” with width σ centered about μ. We have not defined p(s) very rigorously—it is simply the probability of being displaced s from the origin at the end of the K-step random walk. An exact formula for p(s) involves counting all the ways that ±1 can be added K times to get exactly s. We will postpone this, and the proof of CLT, for next time.

Remarkably the CLT applies to any N with any discrete step distribution {p_{i}} and any step displacements {f_{i}}. It also applies to the case of continousvalued steps with distribution p(f) along the real line. However a criterion for validity is always the finiteness of the second moment of p(f).