• Paper: http://www.econ.ucsb.edu/~tedb/Courses/Ec100C/galeshapley.pdf
  • Note on the subsequent Nobel prize! http://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/2012/press.html
• Wikipedia entry on Stable marriage problem: http://en.wikipedia.org/wiki/Stable_marriage_problem
• There's also the marriage / secretary problem: http://en.wikipedia.org/wiki/Secretary_problem

• Induction: http://en.wikipedia.org/wiki/Mathematical_induction
• Strong induction: http://en.wikipedia.org/wiki/Mathematical_induction#Complete_induction (sometimes called complete induction)
• Difference between the two: http://math.stackexchange.com/questions/517440/whats-the-difference-between-simple-induction-and-strong-induction
• Transfinite induction: http://en.wikipedia.org/wiki/Transfinite_induction

• Article: http://www.theatlantic.com/national/archive/2014/09/the-geography-of-nfl-fandom/379729/?utm_source=taboola
• Map: http://cdn.theatlantic.com/assets/media/img/mt/2014/09/Screen_Shot_2014_09_05_at_4.40.46_PM/original.png

• Article: http://www.theatlantic.com/technology/archive/2014/03/here-is-every-us-countys-favorite-baseball-team-according-to-facebook/359917/
• Map: http://cdn.theatlantic.com/assets/media/img/mt/2014/03/Facebook_MLB_Fandom_Map_1/original.jpg

• Fair division is a very important, very hard problem: http://en.wikipedia.org/wiki/Fair_division
• From fair division you can get to voting theory -- how much of a vote should everyone get? This has been an important problem since the discussions of the US Constitution (and much earlier!).

• Lecture notes on the proof are pages 9-11 of this pdf file: http://web.williams.edu/Mathematics/sjmiller/public_html/209/209LectureNotes.pdf
• Wikipedia page: http://en.wikipedia.org/wiki/Picard%E2%80%93Lindel%C3%B6f_theorem
• Sperner's lemma: http://en.wikipedia.org/wiki/Sperner's_lemma
• Mean value theorem: http://en.wikipedia.org/wiki/Mean_value_theorem

• Integral equations: http://en.wikipedia.org/wiki/Integral_equation
• Functional analysis: http://en.wikipedia.org/wiki/Functional_analysis

• Cauchy sequence: http://en.wikipedia.org/wiki/Cauchy_sequence
• Geometric series formula: http://en.wikipedia.org/wiki/Geometric_series
• Ratio test: http://en.wikipedia.org/wiki/Ratio_test
• Root test: http://en.wikipedia.org/wiki/Root_test
• Comparison test: http://en.wikipedia.org/wiki/Limit_comparison_test
• Integral test: http://en.wikipedia.org/wiki/Integral_test_for_convergence

• Least upper bound property: http://en.wikipedia.org/wiki/Least-upper-bound_property (needed to show if \(\{a_n\}\) is a sequence of non-negative terms whose sum is bounded then the sum converges).
• Some important types of spaces (great places to search for contraction maps):
  • Metric space: http://en.wikipedia.org/wiki/Metric_space
  • Normed vector space: http://en.wikipedia.org/wiki/Normed_vector_space
  • Inner product space: http://en.wikipedia.org/wiki/Inner_product_space

• Beautiful clip from the movie War of the Roses: https://www.youtube.com/watch?v=aI5yFnWK9Eg (start at 1:19); script is available here: http://www.script-o-rama.com/movie_scripts/w/war-of-the-roses-script-transcript.html (start reading at Oliver, my father used to say).

• Videos: Mandelbrot zoom: video 1, video 2. Here's a cubic fractal zoom. (other copies: Videos: Mandelbrot set        Newton Fractal (cubic) )
• Mathematica notebook for Newton's Method (pdf version here)

• We spent a decent amount of time looking at the run-time of the Insertion Algorithm. We did this by showing the number of possibilities was essentially \(\sum_{k=0}^n k(n-k)\). While we can (and you should!) evaluate this directly, we can get upper and lower bound estimates without too much trouble.
  • Clearly it's at most \(n^3\) as we have essentially \(n\) terms that are at most \(n^2\) each. We can improve this to \(n^3/4\) as the largest product is when \(k=n/2\).
  • We can get a lower bound by looking at just the middle term. Unfortunately this gives \(n^2/4\), and this is lower order in \(n\). As there is thus a large difference between the upper and lower bounds, we think harder. We go from \(k = n/4\) to \(3n/4\). There are \(n/2\) terms, and each product is at least \(n/4 \cdot 3n/4 = 3n^2/16\), giving a lower bound of 3n^3/32\). Note this is comparable to our upper bound of \(n^3/4 = 8n^3/32\); we're within a factor of 3 (which is great!) with very little work!

• Click here for my notes on the Intermediate Value Theorem: http://web.williams.edu/Mathematics/sjmiller/public_html/BrownClasses/90/MVT_TaylorSeries.pdf

• History of colonial furniture: https://www.onekingslane.com/info/time-period/colonial-american-furniture/ (lot of excellent items)

• history of candle sconces: http://encyclopedia.jrank.org/articles/pages/cmgd0enjaj/Wall-Candle-Sconces-Short-History-Wall-Candle-Sconces-How-They're-Used-Today.html
• another history: http://www.candlewallsconces.net/history-of-candle-wall-sconces/
• from ebay: http://www.ebay.com/gds/What-Are-Sconces-/10000000178706328/g.html
• short Wikipedia page on sconce: http://en.wikipedia.org/wiki/Sconce_(light_fixture)
• finding focal points: http://www.optics4kids.org/home/content/other-resources/articles/lenses-and-geometrical-optics/

• This might be the solution: http://www.woodweb.com/knowledge_base/Measuring_Logs_and_Lumber.html
• Images: https://www.google.com/search?q=log+ruler+logging&espv=2&biw=1366&bih=639&tbm=isch&tbo=u&source=univ&sa=X&ei=iLdOVKDbEc34yQSWpoKIBA&ved=0CDwQsAQ
  • Good image of log ruler: http://takeadiptools.com/attachments/Image/MAINE_LOGGING_TOOLS/saco_parsons_rule2.jpg
  • Good image of people with log ruler: http://www.adirondackalmanack.com/wp-content/uploads/2014/04/log-measurement-1900_01.jpeg

• Parameters: Cities \(C = \{0, 1, 2, \dots, n, n+1\}\). Here 0 is the special start city and n+1 the special end city.
• Parameters: Accessible cities from city i are \(S_i = \{c_{i_1}, \dots, c_{i_{r_i}}\}\). Make sure all are accessible from 0 and all go to n+1.
• Parameters: \(p_{ij}\): cost of traveling from city i to city j; we set \(p_{0j} = 0, p_{i,n+1} = 0\).
• Parameters: Time step: \(t \in \{-1, 0, \dots, n\}\) (so at time -1 leave from the special city to our starting point and then end at a special city at time n+1).
• Parameters: \(p_{ij}\):  cost of travel from city i to city j.
• Variable: \(x_{tij}\) is 1 if at time t we leave city i for city j and 0 otherwise.
• Constraints: First we handle the two special cities, then no longer need to worry about them.
  • Constraint: \(\sum_t \sum_j x_{tj0} = 0\) (this means we never travel to city 0.
  • Constraint: \(\sum_{t \le n-1} \sum_i x_{ti,n+1} = 0\) (this means we do not travel to city n+1 before time n\).
  • Constraint: \(\sum_i x_{n,i,n+1} = 1\) (this means we travel to city n+1 at time n).
• Constraint: for all times \(t \in \{0, \dots, n-1\}\), \(\sum_{i,j \in \{1,\dots,n\}} x_{tij} = 1\); this means at all times of travel we go to exactly one city.
• Constraint: for \(j \in \{1,\dots,n\}\): \(\sum_{t \in \{-1, \dots, n-1\}} \sum_i x_{tij} = 1\): each city j is reached exactly once.
• Constraint: for \(t \in \{-1, \dots, n-1\}\): IF (\(\sum_j x_{tij}= 1\)) THEN (\(\sum_{k \in S_j} x_{t+1,j,k} = 1\)); this  means if we leave at time t for city j, then at time t+1 we must leave city j for one of its accessible cities. Note how useful it is to have our IF-THEN formulation!
• Objective function: minimize \(\sum_t \sum_{i,j} p_{ij} x_{tij}\).

• Overview of exact and approximate solns: http://thesai.org/Downloads/Volume2No1/Paper%204-A%20Genetic%20Algorithm%20for%20Solving%20Travelling%20Salesman%20Problem.pdf
• Genetic algorithm for solving: http://thesai.org/Downloads/Volume2No1/Paper%204-A%20Genetic%20Algorithm%20for%20Solving%20Travelling%20Salesman%20Problem.pdf
• Ant colony approach: http://www.ijiet.org/papers/67-R052.pdf
• Thermodynamic approach: http://www.webpages.uidaho.edu/~stevel/565/literature/tsp.pdf
• Survey on the TSP: http://micsymposium.org/mics_2010_proceedings/mics2010_submission_51.pdf
• Readable history and results: http://cs.indstate.edu/~zeeshan/aman.pdf

• List of NP-complete problems: http://en.wikipedia.org/wiki/List_of_NP-complete_problems
  • Great practice problem for the midterm: solve this game as linear programming: http://en.wikipedia.org/wiki/Battleship_(puzzle)
• P vs NP: http://en.wikipedia.org/wiki/P_versus_NP_problem

• A nice theoretical example of where weights help, in pure mathematics, is in counting primes. The reason for this is the Riemann zeta function \(\zeta(s) = \sum_{n=1}^\infty 1/n^s = \prod_p (1 - p^{-s})^{-1}\) (this follows from the fundamental theorem of arithmetic and the geometric series formula, where \(p\) ranges over the primes); here \(s = \sigma + it\) with \(\sigma > 1\). Whenever we see a product we should try taking logarithms; in this case taking the logarithmic derivative gives \(\zeta'(s)/\zeta(s) = - \sum_{n=1}^\infty \Lambda(n)/n^s\), where \(\Lambda(n) = \log p\) if \(n = p^k\) for some prime \(p\) and 0 otherwise. The reason this is useful is that contour integration in complex analysis works wonderfully with the logarithmic derivative, and this allows us to get information about the primes weighted by their logarithms. Another way of looking at this is that we introduce a slight bias, a small discrimination, among the primes.

• Wikipedia page on knapsack problem: http://en.wikipedia.org/wiki/Knapsack_problem
• Notes on knapsack and dynamic programming: http://www.es.ele.tue.nl/education/5MC10/Solutions/knapsack.pdf
• More dynamical programming (code/constraints) and knapsack: http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/Dynamic/knapsackdyn.htm
• Huge set of notes on knapsack problem: http://www.diku.dk/~pisinger/95-1.pdf

• The Kepler Conjecture (sphere packing). See also T. Hales. See also more general packing problems.

We can use truth tables to convert an IF-THEN statement to an INCLUSIVE OR. Read about Boolean algebras for more on this important topic. You've seen this at various points in your math career; sometimes it's easier to attack the contrapositive rather than the original statement.

A major theme of today's lecture was building complicated functions out of simpler ones. In the end, we could get trunctations, maximum and minimums, and absolute values from IF-THEN and other operations. This shows that we can build complicated functions out of simpler pieces. Some of these we haven't done yet but will either do on Monday, or I'll just leave to the books.

In calculus the absolute value function wreaks havoc, as it is not differentiable. In linear programming it's effect is far less, as we can essentially linearize it (at the cost of introducing binary indicator variables). This can lead to conversations on how to measure errors. The Method of Least Squares is one of my favorites in statistics (click here for the Wikipedia page, and click here for my notes). The Method of Least Squares is a great way to find best fit parameters. Given a hypothetical relationship y = a x + b, we observe values of y for different choices of x, say (x1, y1), (x2, y2), (x3, y3) and so on. We then need to find a way to quantify the error. It's natural to look at the observed value of y minus the predicted value of y; thus it is natural that the error should be Sum_{i=1 to n} h(yi - (a x_i + b)) for some function h. What is a good choice? We could try h(u) = u, but this leads to sums of signed errors (positive and negative), and thus we could have many errors that are large in magnitude canceling out. The next choice is h(u) = |u|; while this is a good choice, it is not analytically tractable as the absolute value function is not differentiable. We thus use h(u) = u²; though this assigns more weight to large errors, it does lead to a differentiable function, and thus the techniques of calculus are applicable. We end up with a very nice, closed form expression for the best fit values of the parameters.

It is possible to get so caught up in reductions and compactifications that the resulting equation hides all meaning. A terrific example is the great physicist Richard Feynman's reduction of all of physics to one equation, U = 0, where U represents the unworldliness of the universe. Suffice it to say, reducing all of physics to this one equation does not make it easier to solve physics problems / understand physics (though, of course, sometimes good notation does assist us in looking at things the right way). I'm mentioning this here as what he does involves measuring errors by squaring, the topic discussed a moment ago. For each physics equation look at the square of the left hand side minus the right hand side, then sum everything and call that U. Thus one term is say (F - ma)^2, and thus we see that the only way U = 0 is if each summand is zero, and thus each physics equation must hold. Suffice it to say, reducing all of physics to this one equation does not make it easier to solve physics problems / understand physics (though, of course, sometimes good notation does assist us in looking at things the right way).

Video online here: http://youtu.be/TPDlEMpXBjg

(Repeating from last Wednesday's additional comment, as it's been awhile): The Simplex Method allows us to solve the standard linear programming problem. There were a lot of clever ideas in the proof. The first was that we used Phase II to prove Phase I and then used Phase I to prove Phase II; this seems illegal as Phase II requires Phase I, but fortunately it isn't. The idea is if we can find a solution to a related problem, we can pass from that to a solution to the problem we care about. This is somewhat similar to the auxiliary lines that appear in geometry proofs; the difficulty is figuring out where to draw them. We needed to pass from our original problem to a related one.

• Instead of taking a feasible solution to the Dual Problem, we took a candidate for the feasible solution to the dual problem. Our candidate involved \(A_B\) (the truncated matrix) and \(b\) and \(c\), so it was definitely a reasonable choice. We then showed that if this candidate y were feasible then we had an optimal solution to the original problem.
• The difficulty is what happens if our candidate is not feasible? We then had a constructive procedure on what to do next, specifically, what column to swap out. We're trying to minimze cost, and this told us where to move. We then continued, got a new feasible solution, and saw we could either send the cost all the way to negative infinity or we'd eventually terminate in an optimal solution. The reason is we can't run through the procedure infinitely many times because there are only finitely many basic feasible solutions, and we had a basic feasible solution.
• We now see how all our different assumptions come to play. We use the canonical formulation to ensure things are in a certain form. We needed b to be non-negative in proving Phase I, and since we eliminated inequalities we could multiply rows by -1 if needed and make this happen. The final key idea is similar to Fermat's Proof by Infinite Descent. There are only finitely many basic feasible solutions, so we cannot keep looping forever; eventually the procedure must terminate! I have a very large appendix on proof techniques; if you're interested email me and I'll send it, if not a few are posted here.

• There are many processes in mathematics that run in far less time than they appear. Below is a quick description of a few, many of which we've already seen (but this is a good time to revisit). The Euclidean algorithm is a terrific example. It runs far faster than expected; it doesn't take \(\min(x,y)\) steps but rather \(2 \log(\min(x,y))\). It's a beautiful example of a major theme of the course, the need to do things fast. The other topic was linearization non-linear terms. This was a major theme of calculus (Newton's method is a terrific example). Here are some other examples.
  • Horner's algorithm to evaluate a polynomial quickly: \(4x^3 + 5x^2 - 3x + 8 = ((4x+5)x - 3)x + 8\) (saves a few multiplications!). Saving multiplications is very important; one application of evaluating polynomials quickly is in constructing Mandelbrot sets (see below when we discuss Newton's method).
  • Telescoping sums: often re-arranging the algebra leads to a significantly easier computation.
  • Fast matrix multiplication: Naively we expect it takes \(N^3\) multiplications to find all \(N^2\) entries of \(A^2\) or \(AB\) when \(A\) and \(B\) are \(N\times N\) matrices. The Strassen algorithm (see also the Mathworld entry here, which I think is a bit more readable) does it in about \(N^{\log_2 7}\); the reason for this savings is that they can multiply two \(2\times 2\) matrices with \(7\) and not \(8\) multiplications (\(3 = \log_2 8\)). The best known algorithm is the Coopersmith-Winograd algorithm, which is of the order of \(N^{2.376}\) multiplications. See also this paper for some comparison analysis, or email me if you want to see some of these papers.
    • Some important facts. The Strassen algorithm has some issues with numerical stability.
    • One can ask similar questions about one dimension matrices, ie, how many bit operations does it take to multiply two \(N\) digit numbers. It can be done in less than \(N^2\) bit operations (again, very surprising!). One way to do this is with the Karatsuba algorithm (see also the Mathworld entry for the Karatsuba algorithm).
  •  Newton's method is significantly more powerful than divide and conquer (also called the bisecting algorithm); this is not surprising as it assumes more information about the function of interest (namely, differentiability). The numerical stability of Newton's method leads to many fascinating problems. One terrific example is looking at roots in the complex plane of a polynomial. We assign each root a different color (other than purple), and then given any point in the complex plane, we apply Newton's method to that point repeatedly until one of two things happen: it converges to a root or it diverges. If the iterates of our point converges to a root, we color our point the same color as that root, else we color it purple. This leads to Newton fractals, where two points extremely close to each other can be colored differently, with remarkable behavior as you zoom in. If you're interested in more information, let me know; a good chaos program is xaos (I have other links to such programs for those interested). One final aside: it is often important to evaluate these polynomials rapidly; naive substitution is often too slow, and Horner's algorithm is frequently used. What we're really doing in Newton's method is replacing a complicated function with a linearization.
    • One of the most important applications of the Euclidean algorithm is in RSA (cryptography) to compute modular inverses, and to determine some of the items needed for our keys.
    • There were several great questions from the class about the efficiency of the Euclidean algorith. We initially looked at it and said it must run in at most \(x\) steps (where \(x < y\) are our two numbers). After some analysis we showed it runs in at most \(2 \log_2(x)\) steps; however, maybe it's even faster, or faster on average. Click here for more on it's runtimes, and the various measures (average run time, worst case scenario). This leads to the topic of computational complexity.