A PDF copy of the syllabus can be found here: APMA_1200.pdf

Apr 18 - Apr 23: We introduced the Galton-Watson process; we investigated the extinction probability and derived a large deviation principle for the total population size in the subcritical regime.

The fifth homework assignment has been posted.

Apr 4 - Apr 16: We looked at examples of Markov decision processes, and investigated the existence of optimal stationary policies (section 5.4).

April 2: We analyzed various betting strategies, including the martingale strategy, and we proved the optional sampling theorem.

The fourth homework assignment has been posted.

March 19: We defined the notion of a martingale, and proved that sub/super-harmonic functions of Markov chains generate sub/super-martingales (section 4.1).

March 14: We focused on potential theory and proved that Markov chain potentials satisfy a discrete Poisson equation with appropriate boundary conditions (theorem 4.2.3).

March 7: We introduced the Doeblin condition and derived some basic results on geometric ergodicity.

March 5: We discussed some elements of ergodic theory and proved the mean ergodic theorem for irreducible and positive recurrent Markov chains (theorem 1.10.2).

The third homework assignment has been posted.

Feb 21 - Feb 26: We proved that every irreducible and recurrent stochastic matrix has an essentially unique positive invariant measure. We also investigated the relation between invariant distributions and expected return times in the context of positive recurrent chains (theorems 1.7.6 and 1.7.7).

Feb 14 - Feb 19: We made use of the strong Markov property to prove the transience/recurrence dichotomy for Markov chains (section 1.5).

February 12: We defined the notion of a stopping time and introduced the strong Markov property (section 1.4).

The second homework assignment has been posted.

Feb 5 - Feb 7: We discussed the first-step analysis approach to computing absorption probabilities and mean hitting times (theorems 1.3.2 and 1.3.5).

January 31: We covered section 1.2 on communicating classes and irreducible chains.

The first homework assignment has been posted.

January 29: We discussed the computation of multi-step transition probabilities, and we derived the Chapman-Kolmogorov equation.

January 24: We defined the notion of a Markov chain and focused on specific examples, such as nearest neighbor random walks and the gambler's ruin problem.