Honors 1: Undergraduate Math Lab^{∗}

Peter Sarnak^{†}, Steven Miller^{‡}, Alex Barnett^{§}

October 2nd, 2002

Courant Institute of Mathematical Sciences New York University New York, NY

Abstract

Proof of the Central Limit Theorem (via the Fourier Transform). Proof of the Spectral Theorem for Real Symmetric Matrices (via maximization). Spectral Gap and Families of Expanders. Lecture by Peter Sarnak; notes by Steven Miller.

1 Central Limit Theorem

Let X : Ω → ℝ be a random variable (x(ω) ∈ ℝ). Define the density function μ on ℝ by

μ[a, b] = Prob(ω : x(ω) ∈ [a, b]).  (1)

Thus, X gives rise to the probability measure μ on ℝ.

Assume

1. X_{1}, X_{2}, … are independent identically distributed random variables (iidrv) with density μ.

∗Homepage: http://www.math.nyu.edu/∼millerj/

†E-mail: sarnak@math.princeton.edu

‡E-mail: millerj@cims.nyu.edu or sjmiller@math.princeton.edu

§E-mail: barnett@nmr.mgh.harvard.edu

2. E(X) = ∫_{Ω} x(ω)dp(ω) = ∫_{ℝ} xdμ(x).

3. Var(X) = ∫_{Ω} x^{2}(ω)dp(ω) = ∫_{R} x^{2}dμ(x) = 1.

Define S_{N} = ∑ X_{j}, and let N(0, 1) denote the standard Gaussian or normal distribution (mean zero, variance one).

Theorem 1.1.

\frac{S_{N}}{\sqrt{N}} \overrightarrow{in probability} N(0, 1),  (2)

ie,

Prob(\frac{S_{N}}{\sqrt{N}} ∈ [a, b]) → \frac{1}{\sqrt{2π}} ∫_{a}^{b} e^{− \frac{x^{2}}{2}} dx.  (3)

Proof: We have a random variable X, induces a probability measure μ on the real line ℝ. Thus, dμ = f(x)dx, f(x) is a nice function. As μ is a probability measure on ℝ, we must have f(x) ≥ 0 and ∫_{R} f(x)dx = 1.

The given assumptions about the X_{i}s imply

1. ∫_{R} xf(x)dx = 0.

2. ∫_{R} x^{2}f(x)dx = 1.

Definition 1.2 (Fourier Transform).

f̂(ξ) = ∫_{R} f(x)e^{−2πixξ}dx.  (4)

Clearly, |f̂(ξ) ≤ ∫_{R} f(x)dx ≤ 1. Further, f̂(0) = ∫_{R} f(x)dx = 1. Now

f̂′(ξ) = ∫_{R}(2πix)f(x)e^{−2πixξ}dx.  (5)

Thus, f̂′(0) = 0 (from E(x) = 0). We will assume f̂ is continuous (although this is implied by our assump-

tions).

Further,

f̂′′(ξ) = −4π^{2} ∫_{R} x^{2}f(x)e^{−2πixξ}dx.  (6)

Therefore, f̂′′(0) = −4π^{2} (by our assumption on the variance).

Using Taylor to expand f̂ we obtain

f̂(ξ) = 1 + \frac{f′′(0)}{2}ξ^{2} + ⋯
= 1 − 2π^{2}ξ^{2} + ⋯  (7)

Near the origin, the above shows f̂ looks like a concave down parabola.

Theorem 1.3 (Fourier Inversion). If f is a nice function (so all the quantities below make sense, ie, f decays fast enough):

f(x) = ∫_{R} f̂(ξ)e^{2πixξ}dξ.  (8)

Definition 1.4. e(y) = e^{2πiy}.

Exercise 1.5. If ϕ_{ξ}(x + y) = e((x + y)ξ), prove ϕ_{ξ}(x + y) = ϕ_{ξ}(x)ϕ_{ξ}(y). ϕ_{ξ} is a character of (ℝ, +).

Exercise 1.6. Is there a ψ : ℝ → ℝ such that ψ(x + y) = ψ(x) + ψ(y)? IE, can you find a homomorphism that takes addition to addition? If yes (of course: see above!) what can you say about ψ? If we assume ψ is continuous, it must be of the form ϕ_{ξ}.

Think of ℝ as a vector space over ℚ (ie, the scalars are ℚ). What is the dimension of ℝ over ℚ, and what is a basis? As the reals are uncountable and the rationals are countable, there are uncountably many basis vectors.

Any linear transformation will satisfy the desired condition! For example, choose any basis (called a Hamel Basis) of ℝ over ℚ (very hard! need the Axiom of Choice).

To show every character of the reals is of the form ϕ_{ξ}, you need more. If you assume the character is continous, then it must be of the form ϕ_{ξ}.

Suppose we have random variables X and Y with measures μ and ν, with induced functions f(x) and g(x). If we choose X and Y independently, what is the distribution of X + Y?

Lemma 1.7. The distribution of X + Y clearly cannot be the sum (as that won’t be a probability measure). It is f ∗ g, and f ∗ g is a probability measure.

Definition 1.8 (Convolution).

(f ∗ g)(x) = ∫_{R} f(x − y)g(y)dy  (9)

We show the convolution is a probability measure. We assume our functions f and g are nice (ie, that we may use Fubini to interchange the order of integration).

∫_{R}(f ∗ g)(x)dx = ∫_{R} ∫_{R} f(x − y)g(y)dydx
= ∫_{R} ∫_{R} f(x − y)g(y)dxdy
= ∫_{R} g(y)( ∫_{R} f(x − y)dx)dy

= ∫_{R} g(y)( ∫_{R} f(t)dt)dy
= ∫_{R} g(y) · 1dy = 1.  (10)

Exercise 1.9. f ∗ g^{}(ξ) = f̂(ξ) · ĝ(ξ). Thus, Fourier Transform converts convolution to multiplication.

Let Z = X + Y. What is

Prob(Z ∈ [z, z + dz]) = h(z)dz.  (11)

x can be anywhere; given x, y must lie between z − x and z − x + dz.

Prob(X + Y ∈ [z, z + dz]) = ∫_{x∈ℝ} Prob(X ∈ [x, x + dx] and

Y ∈ [z − x, z − x + dz])f(x)dx

= ∫_{x∈ℝ} f(x)g(z − x)dx = h(z),  (12)

where the last step follows from the definition of the density functions.

By Induction, we see X_{1} + ⋯ X_{N} has distribution f ∗ ⋯ ∗ f (as the random variables are iidrv).

However, we want to study S_{N} = \frac{X_{1}+···X_{N}}{\sqrt{N}}.

Definition 1.10 (FT). Let F T (f) = f̂; F T denotes the Fourier Transform.

Lemma 1.11.

F T (the distribution of \frac{X_{1} + ⋯ + X_{N}}{\sqrt{N}} ) = [f̂(\frac{ξ}{\sqrt{N}} )]^{N}. (13)

We take the limit as N → ∞ for fixed ξ. Recall we showed that f̂(ξ) = 1 − 2π^{2}ξ^{2} + · · ·. Thus, we have to study

[1 − \frac{2π^{2}ξ^{2}}{N} + O(\frac{|ξ|^{3}}{N^{\frac{3}{2}}} )]^{N}.  (14)

But as N → ∞, the above goes to

e^{−2πξ^{2}}.  (15)

The universality arises because only terms up to the second order in the Taylor Series contribute.

Exercise 1.12. Show the Fourier Transform of the Gaussian is the Gaussian.

Key point:

• Used Fourier Analysis to study the sum of independent identically distributed random variables, as it converts convolution to multiplication.

Exercise 1.13. Fix g a nice, smooth, rapidly decreasing function. Consider the linear transformation A:

(Af)(x) = ∫_{R} g(x − y)f(y)dy.  (16)

Any convolution operator is diagonalized by these characters (exponentials). At a formal level,

(Aϕ_{ξ})(x) = ∫_{R} g(x − y)ϕ_{ξ}(y)dy
= ∫_{R} g(x − y)e^{−2πiξy}dy
= ∫_{R} g(t)e^{−2πiξ(x−t)}dt
= e^{−2πixξ} ∫_{R} g(t)e^{2πiξt}dt = ĝ(ξ)ϕ_{ξ}(x).  (17)

Thus, ϕ_{ξ} is an eigenvector of A with eigenvalue ĝ(ξ).

2 Spectral Theorem for Real Symmetric Matrices

Let A be a real symmetric matrix acting on ℝ^{n}. Then A has an orthonormal basis v_{1}, …, v_{n} such that Av_{j} = λ_{j}v_{j}.

A simpler proof, assuming all eigenvalues are distinct, is available in the September 25^{th} lecture notes.

Write the inner or dot product ⟨v, w⟩ = v^{t}w. As A is symmetric, ⟨Av, w⟩ = ⟨v, Aw⟩.

Definition 2.1. V^{⊥} = {w : ∀v ∈ V, ⟨w, v⟩ = 0}.

Lemma 2.2. Suppose V ⊂ ℝ^{n} is an invariant vector subspace under A (ifv ∈ V, then Av ∈ V). Then V^{⊥} is also A-invariant: A(V^{⊥}) ⊂ V^{⊥}.

This proves the spectral theorem. Suppose we find a v_{0} ≠ 0 such that Av_{0} = λ_{0}v_{0}. Take V = {μv_{0} : μ ∈ ℝ} for the invariant subspace.

By Lemma 2.2, V^{⊥} is left invariant under A, and is one dimension less. Thus, by whatever method we used to find an eigenvector, we apply the same method on V^{⊥}.

Thus, all we must show is given an A-invariant subspace, there is an eigenvector.

Consider

max_{v with ⟨v,v⟩=1} {⟨Av, v⟩}.  (18)

Standard fact: every continuous function on a compact set attains its maximum (not necessarily uniquely). See, for example, W. Rudin, Principles of Mathematical Analysis.

Let v_{0} be a vector giving the maximum value, and denote this maximum value by λ_{0}. As ⟨v_{0}, v_{0}⟩ = 1, v_{0} is not the zero vector.

Lemma 2.3. Av_{0} = λ_{0}v_{0}.

Clearly, if Av_{0} is a multiple of v_{0} it has to be λ_{0} (from the definition of v_{0} and λ_{0}).

Thus, it is sufficient to show

Lemma 2.4. {μv_{0} : μ ∈ ℝ} is an A-invariant subspace.

Proof: let w be an arbitrary vector perpendicular to v_{0}, and ϵ be an arbitrary small real number. Consider

⟨A(v_{0} + ϵw), v_{0} + ϵw⟩  (19)

We need to renormalize, as v_{0} + ϵw is not unit length; it has length 1 + ϵ^{2}⟨w, w⟩. As v_{0} was chosen to maximize ⟨Av, v⟩ subject to ⟨v, v⟩ = 1, after normalizing the above cannot be larger. Thus,

⟨A(v_{0} + ϵw), v_{0} + ϵw⟩ = ⟨Av_{0}, v_{0}⟩ + 2ϵ⟨Av_{0}, w⟩ + ϵ^{2}⟨w, w⟩. (20)

Normalizing the vector v_{0} + ϵw by its length, we see that in Equation 20, the order ϵ terms must be zero. Thus,

⟨Av_{0}, w⟩ = 0;  (21)

however, this implies Av_{0} is in the space spanned by v_{0} (as w was an arbitrary vector perpendicular to v_{0}), completing our proof. □

Corollary 2.5. Any local maximum will lead to an eigenvalue-eigenvector pair.

The second largest eigenvector (denoted λ_{1}) is

λ_{1} = max_{⟨v,v_{0}⟩=0} \frac{⟨Av, v⟩}{⟨v, v⟩}.  (22)

We can either divide by ⟨v, v⟩, or restrict to unit length vectors.

3 Applications to Graph Theory

Let G be a k-regular graph, f : V → ℝ. Recall v ∼ w if there is an edge connecting v and w. Let A be the adjacency matrix of G, and define

Af(v) = ∑_{v∼w} f(w)
⟨f, g⟩ = ∑_{v∈V} f(v)\overline{g}(v).  (23)

Consider the function f_{0}(v) = 1 for all v ∈ V. Then

Af_{0}(v) = ∑_{v∼w} f_{0}(w) = kf_{0}(v).  (24)

Thus, f_{0} is an eigenfunction with eigenvalue k.

Theorem 3.1 (Expander Families). Fix k. Suppose we have a sequence of k-regular graphs G_{j} with |V_{j}| → ∞ and suppose k − λ_{1}(G) ≥ δ > 0, δ fixed. Then G_{j} is an expander family.

Remark 3.2. If you have an algorithm that has a random element, then one can show there is another algorithm which does what this algorithm does without having a random component. (More or less, some slight of hand).

Suppose we have a bipartite graph: there are two sets of vertices I (inputs) and O (outputs). Edges run only between I and O; there are no edges between two vertices in I or between two vertices in O. Let there be n inputs and n outputs, and join each input with k outputs.

Fix δ_{0} > 0. We want, for any B ⊂ I, |∂B| ≥ δ_{0}|B|.

We give a sketch of the proof. We will show that knowledge of a spectral gap ensures that the boundary of any subset of I will be big. We do bipartite for simplicity.

Let B ⊂ I. Define

f(v) = \begin{cases}2n − |B| & for v ∈ B \\ −|B| & otherwise. \end{cases}  (25)

Then

∑_{v∈V} f(v) = |B| · (2n − |B|) + (2n − |B|) · (−|B|) = 0.  (26)

Then

λ_{1} = max_{⟨f̃,f_{0}⟩=0} = \frac{⟨Af̃, f̃⟩}{⟨f̃, f̃⟩}.  (27)

In particular,

\frac{⟨Af_{1}, f_{1}⟩}{⟨f_{1}, f_{1}⟩} ≤ λ_{1}.  (28)

Definition 3.3 (Laplacian). Δ = kI − A.

Thus, Δf_{0} = kf_{0} − Af_{0} = 0 · f_{0}. The eigenvalues of Δ are trivially related to the eigenvalues of A.

Remark 3.4 (Motivation for Laplacian). On the line we have \frac{d^{2}}{dx^{2}} . A discrete version is (exercise)

\frac{f(x + h) − 2f(x) + f(x − h)}{h^{2}}.  (29)

In the plane, we would have \frac{d^{2}}{dx_{1}^{2}} + \frac{d^{2}}{dx_{2}^{2}} . Integrating by parts we have

∫_{Ω}(Δf) · gdx_{1}dx_{2} = − ∫_{Ω} ∇f · ∇gdx_{1}dx_{2}
= ∫_{Ω} f · (Δg)dx_{1}dx_{2}.  (30)

We want to integrate by parts on a graph!

(ΔF)(x) = kF (x) − ∑_{x∼y} F (y).  (31)

Therefore

⟨ΔF, F ⟩ = ∑_{x∈V} (kF (x) − ∑_{y∼x} F (y))F (x)
= k ∑_{x∈V} F (x)^{2} − ∑_{x∈V} ∑_{x∼y} F (x)F (y).  (32)

For each edge e, orient it by e^{+} and e^{−}. The analogue of the Laplacian becomes

∑_{e} (F (e^{+}) − F (e^{−}))^{2} = ∑_{e} F^{2}(e^{+}) − 2F (e^{+})F (e^{−}) + F^{2}(e^{−})
= 2⟨ΔF, F ⟩,  (33)

where the last line follows by thinking about what it means for vertices to be connected, and how often each vertex is hit.

Recall

k − λ_{1} = min_{⟨F,f_{0}⟩=0} \frac{⟨ΔF, F ⟩}{⟨F, F ⟩} = min_{⟨F,f_{0}⟩=0} \frac{1}{2} \frac{||dF ||^{2}}{⟨F, F ⟩} .  (34)

Plug in the function f defined above, namely,

f(v) = \begin{cases}2n − |B| & for v ∈ B \\ −|B| & otherwise. \end{cases}  (35)

If the edge runs from input to input or output to output, we get zero. The only way we get non-zero contribution is from an input to an output (or vice-versa). For our f,

\frac{1}{2} ∑_{e∈E} |f(e^{+}) − f(e^{−})|^{2} = \frac{1}{2}(2n)^{2} · #\{edges e running from B to B^{c}\}, (36)

where B^{c} is the complement of B. We want 0 < δ_{0} = k − λ_{1}. Divide the previous equation by ⟨F, F ⟩, where ⟨F, F ⟩ = (2n−|B|)^{2}·|B|+|B|^{2}·(2n−|B|) = |B| · (2n − |B|) · 2n.

Thus,

δ_{0} ≤ \frac{\frac{1}{2}(2n)#\{edges\}}{|B| · (2n − |B|) · 2n}.  (37)

Therefore,

#\{edges\} ≥ \frac{δ_{0}|B| · (2n − |B|)}{n}
≥ δ_{0}|B|,  (38)

as |B| ≤ n.

Thus, the total number of edges is at least δ_{0}|B|. But each vertex gets k edges. Thus,

|∂B| ≥ \frac{δ_{0}|B|}{k}.  (39)

4 2\sqrt{k − 1}

Let G be a k-regular connected graph. A is the adjacency matrix. Biggest eigenvalue is λ_{0} = k. Eigenvalues cannot be smaller than −k. How big is the gap between k and λ_{1}?

Theorem 4.1 (Alon-Boppana). Fix k. Take any sequencce of graphs where the number of vertices |G| → ∞. Then

\overline{lim}_{|G|→∞}λ_{1}(A_{G}) ≥ 2\sqrt{k − 1}.  (40)

Remark 4.2. In 1-dimension, with probability one the drunk returns home; same in 2-dimensions. He escapes with finite probability in 3 and higher dimensions!

Consider a 3 regular graph with many vertices. Go to v, look locally. If there are no short circuits, know what it looks like locally: it will look like a tree. (This is p-adic hyperbolic geometry).

Let T be the infinite tree where each vertex is connected to three other vertices (and a vertex cannot be connected to itself). Suppose a drunk is walking on a tree. The only way he can get back is to exactly undo what he’s done.

Consider the following operator: consider an infinite dimensional Hilbert space l_{2}(V), the set of all f : V → ℝ such that ∑_{v} |f(v)|^{2} < ∞. This space is infinite dimensional (for each v, take the function f_{v}(w) = 1 if w = v and 0 otherwise).

Using |ab| ≤ \frac{a^{2}+b^{2}}{2} ,

⟨f, g⟩ = ∑_{v} f(v)g(v)  (41)

exists (and is our inner product). We define

Af(v) = ∑_{w∼v} f(w).  (42)

It is not obvious that there are any eigenvectors (and, in fact, there are no eigenvectors!). There is still a notion of spectrum. We will show the spectrum of this operator (k = 3) is [−2\sqrt{2}, 2\sqrt{2}].

Note the constant function is horrendously not in this space (not even close to being square-integrable.