Spring 2025: Probability Theory II (EN.553.721)

Course Description

This is the second semester of a rigorous introduction to measure-theoretic probability for graduate students. I will assume as a prerequisite that you took Probability Theory I (EN.553.720) in Fall 2024 and are comfortable with that material. You are welcome to attend or enroll if you did not, but you will be responsible for working through any material from there that you are not familiar with.

The main goal of this course is to continue to familiarize you with the essential examples of classical probability theory. We will develop a deep understanding of simple random walks, the most foundational such examples. We will also study their generalizations to have dependent steps (martingales), general domains (Markov chains), and continuous paths (Brownian motion). Along the way, we will learn how to formulate and prove probabilistic statements like convergence results, distributional limit theorems, and concentration inequalities, and will introduce many examples of random structures from different areas of mathematics.

The following is a tentative ordered list of the broad topics we will aim to cover:

  • Central limit theorem
  • Poisson limit theorem and point process
  • Conditional expectation
  • Martingales
  • Markov chains and ergodic theory
  • Brownian motion
  • Basics of stochastic calculus (time permitting)
Contact & Office Hours

Contact information for the instructor of this course (me) and the teaching assistants is below. The best way to contact us is by email. We will decide on a time for office hours in the first week or two of class; in the meantime, please contact us directly to schedule an appointment if you want.

Role Name Email Office Hours
Instructor Tim Kunisky kunisky [at] jhu.edu 3-4pm, Thursdays
TA 1 Debsurya De dde4 [at] jhu.edu 4:30-5:30pm, Tuesdays
TA 2 Yufei Zhan yzhan11 [at] jhu.edu 3-4pm, Fridays (Wyman S425)
Schedule

Class will meet Mondays and Wednesdays, 1:30pm to 2:45pm in Hodson 211.

Below is a tentative schedule, to be updated as the semester progresses.

Date Details
Week 1
Jan 22 General introduction and logistics. Review of ending of Probability Theory I: law of large numbers, characteristic functions, and sketch of Lyapunov's proof of Central Limit Theorem (CLT). [Notes]
Week 2
Jan 27 Review of weak convergence and convergence in distribution. Lindeberg's exchange principle and alternate proof of the CLT. Berry-Esseen quantitative CLT. [Notes]

References:
Jan 29 Brief discussion of further applications of Lindeberg exchange principle. Lindeberg CLT and Bernoulli examples. Rare events and sparse sums of independent random variables: Poisson limit theorem and informal introduction to Poisson point process. [Notes]

References:

  • Klenke, Section 5.5: Good introduction to the one-dimensional Poisson point process.
  • Tao lecture notes on variants of the CLT: Discussion of the relation between the Poisson and central limit theorems and a more general Poisson limit theorem than we proved.
Week 3
Feb 3 Preliminary discussion of nuances of continuous time processes. Revisiting Poisson point process and convergence of Bernoulli process more formally. [Notes]

References:

  • There are good summaries of the measurability issues and restricting the space of sample paths at the beginning of Varadhan's book and, in more detail in the context of Brownian motion, in Section 7.1 of Durrett's book (both linked below).
  • There is more advanced discussion at the beginning of Chapter V of Pollard's book (also linked below), which gives a preview of some other issues we will try to touch on later.
  • George Lowther notes on "versions" of processes: More discussion of these issues and why and how they are dealt with.
Feb 5 Poisson process finite distributions and convergence theorem. Exponential spacings of arrival times. Appearance in extreme value theory. Conditional expectation: brief introduction. [Notes]

References:

  • Durrett Section 3.7 discusses the Poisson point process nicely.
  • A more advanced but readable reference (including on the important topic of Poisson processes not on the line but also in higher-dimensional spaces) is Kingman's book Poisson Processes (which you will have to find yourself).
Week 4
Feb 10 Conditional expectation: definition, examples, existence theorem, main properties. [Notes]

References:

  • We saw a mostly standard modern treatment of this topic; see, e.g., Durrett Section 4.1 for a similar reference.
Feb 12 Finish up properties of conditional expectation. Filtrations and martingales: motivation, definitions, examples. Martingale transform. The martingale gambling strategy. [Notes]
Week 5
Feb 17 (Recorded lecture) Hoeffding concentration inequality for independent sums. Subgaussian random variables. Azuma-Hoeffding inequality for martingales. Bounded differences inequality for nonlinear concentration. Coupon collector / balls-and-bins problem example. [Video] [Blackboard] [Notes]
Feb 19 (Recorded lecture) Stopping times. Doob's optional stopping theorem. Application to distribution of exit time and exit location in simple random walk. [Video] [Blackboard] Note: I kept accidentally saying "optimal stopping theorem" in this recording. The result, as I correctly wrote on the board, is actually called the "optional stopping theorem."
Week 6
Feb 24 Martingale maximal inequalities and convergence theorems.
Feb 26 More applications of martingales. Simple random walk hitting time. Ballot theorem. Branching processes. Hewitt-Savage 0/1 law.
Lecture Notes and Materials

I will post handwritten lecture notes shortly after each of our meetings. I will lecture on the blackboard, so you are encouraged to come to all classes if you want to make sure you are following the material in detail. We will mostly follow a combination of the following books:

Especially later in the course, we will touch on some rather technical issues about constructions and convergence of continuous-time stochastic processes, for which these might be useful further references:

Assignments & Grading

Grades will be based on written homework assignments (roughly every two weeks) and a take-home final exam.

Homework is posted below, and is to be submitted through Gradescope. Further details will be provided when the first assignment is due. Two important points about homework:

  • You are welcome to collaborate with other students on the homework assignments. However, you must write up your own solutions and you may not share your completed solutions with other students. You will receive no credit for a homework assignment if we discover that you have not followed these rules.
  • You may use a total of five late days for homework submissions over the course of the semester without penalty. If you need an extension beyond these, you must ask me 48 hours before the due date of the homework and have an excellent reason. After you have used up these late days, further late assignments will be penalized by 20% per day they are late.

Assignment Assigned Due Links
Homework 1 Jan 29 Feb 10 [PDF]
Homework 2 Feb 14 Feb 28 [PDF] [TeX]