A PDF copy of the syllabus can be found here: APMA_1200.pdf

April 6: We proved that sample paths of Brownian motion are with probability one nowhere differentiable.

The fifth homework assignment has been posted.

April 1: We investigated the asymptotic distribution of $LaTeX: W_n(t)=\frac{S_{\lfloor nt \rfloor}}{\sqrt{n}},$ where LaTeX: S_n is a simple random walk, and concluded the lecture by defining Brownian motion (section 4.4).

Notes: Brownian_motion.pdf

March 30: We defined continuous-time Markov chains and derived the Kolmogorov forward equation (section 2.1).

Notes: continuous-time_Markov_chains.pdf

March 26: We made use of potential theory to prove theorem 5.4.3 on the existence of optimal stationary policies.

March 23: We defined Markov decision processes and introduced the dynamic programming approach to computing optimal policies (section 5.4).

Notes: Markov_decision_processes.pdf

March 16: We focused on potential theory and proved that Markov chain potentials satisfy a discrete Poisson equation with appropriate boundary conditions (section 4.2).

The fourth homework assignment has been posted.

Notes: potential_theory.pdf

March 11: We continued our discussion of the martingale strategy, and proved the optional sampling theorem.

March 9: We defined the notion of a martingale and analyzed the martingale betting strategy. We concluded the lecture by introducing the optional sampling theorem (section 4.1).

Notes: martingales.pdf

March 4: We discussed some elements of ergodic theory and proved the mean ergodic theorem for irreducible and positive recurrent Markov chains (section 1.10).

March 2: We introduced the Doeblin condition and derived some basic results on geometric ergodicity. See, e.g., section 2.2 of Stroock's text on Markov processes

The third homework assignment has been posted.

Notes: p79-p89.pdf

February 25: We discussed the Metropolis algorithm and its applications in discrete optimization.

February 23: We defined the notion of a reversible Markov chain and turned our attention to Markov chain Monte Carlo methods.

Notes: p62-p78.pdf

February 16: No class due to University holiday.

Feb 11 - Feb 18: We proved that every irreducible and recurrent stochastic matrix has an essentially unique positive invariant measure. We also investigated the relation between invariant distributions and expected return times in the context of positive recurrent chains (theorems 1.7.6 and 1.7.7).

The second homework assignment has been posted.

Notes: p50-p61.pdf

February 9: We made use of the strong Markov property to prove the transience/recurrence dichotomy for Markov chains (section 1.5).

February 4: We defined the notion of a stopping time and introduced the strong Markov property (section 1.4).

The first homework assignment has been posted.

Notes: p34-p49.pdf

Jan 28 - Feb 2: We discussed the first-step analysis approach to computing absorption probabilities and mean hitting times (theorems 1.3.2 and 1.3.5).

Notes: p15-p33.pdf

January 26: We derived the Chapman-Kolmogorov equation and discussed the computation of multi-step transition probabilities (section 1.1). We also covered section 1.2 on communicating classes and irreducible chains.

January 21: We defined the notion of a Markov chain and focused on specific examples, such as nearest neighbor random walks and the gambler's ruin problem.

My lecture notes can be found here: p1-p14.pdf