References
Topic 1: The Connection between Probability and Measure Theory
For a discussion of the interplay between "coin tossing", Lebesgue measure on the interval [0,1], and infinite-dimensional measure spaces, see:- Breiman, L. (1968). Probability. Addison-Wesley, Reading, MA (Chapters 1 and 2)
- Billingsley, P. (1979). Probability and Measure. John Wiley, New York (Chapter 1, Sections 1 and 2)
Topic 2: Linear Models
The basic linear regression model, and its multiple regression cousin, is developed in many basic statistics books. See, for example,- Trivedi, K.S. (1982). Probability and Statistics, with Reliability, Queueing, and Computer Science Applications. Prentice Hall, Englewood Cliffs, NJ (Chapter 11, Sections 1 to 7)
for an elementary introduction. The statistical theory of autoregressive processes is described in:
- Anderson, T.W. (1971). The Statistical Analysis of Time Series. John Wiley, New York. (Chapter 5, Sections 1 to 6)
A discussion of state space models and Kalman filtering can be found in the following:
- Hannan, E.J. and Deistler, M. (1988). The Statistical Theory of Linear Systems. John Wiley, New York (Chapter 3)
- Kailath, T. (1976). Lectures on Linear Least-Squares Estimation. Springer-Verlag, New York. (Chapter 5)
- Anderson, B.D.O. and Moore, J.B. (1979). Optimal Filtering. Prentice-Hall, Englewood Cliffs, NJ (Chapter 3)
Gilbert Strang has a very nice (and quite compact) discussion of the Kalman filter in:
- Strang, G. (1986). Introduction to Applied Mathematics. Wellesley-Cambridge Press, Wellesley, MA (Chapter 2, Section 5)
A discussion of prediction theory, and its relation to the geometry of the Hilbert space of square-integrable rvs, can be found in:
- Karlin, S. and Taylor, H.M. (1975). A First Course in Stochastic Processes. Academic Press, New York. (Chapter 9, Sections 1 to 4)
For a relatively elementary introduction to Gaussian processes, see Chapters 4 and 5 of:
- Pg. Hoel, S.C. Port, and C.J. Stone (1972), Introduction to Stochastic Processes, Houghton-Mifflin Company, Boston.
Topic 3: Discrete-Time Markov Chains
The next major topic to be discussed in the course will be Markov chain theory. There are a number of good references here, depending on the level of mathematical sophistication which one is looking for.
An elementary introduction to discrete-time Markov chain theory on discrete state space can be found in:
- S.M. Ross (2003), Introduction to Probability Models, Academic Press, New York (Chapter 4)
For a more complete introduction to the theory of discrete-time Markov chains on discrete state space, see:
- S. Karlin and H.M. Taylor (1975), A First Course in Stochastic Processes. Academic Press, New York (Chapters 2 and 3)
The theory of discrete-time Markov chains on general state space can be found in:
- S.P. Meyn and R.L. Tweedie (1993). Markov Chains and Stochastic Stability. Springer-Verlag, New York.
Topic 4: Stochastic Control
For a good basic introduction to stochastic control in discrete time, see:
- S.M. Ross (1983). Introduction to Stochastic Dynamic Programming. Academic Press.
For a discussion of optimal stopping (as arises, for example, in the context of American options), see:
- E. Cinlar (1975). Introduction to Stochastic Processes. Prentice Hall (Chapter 7)
Topic 5: Continuous-Time Markov Chains (also known as Markov Jump Processes)
For an elementary introduction to the theory of continuous-time Markov chains, see:
- S.M. Ross (2003). Introduction to Probability Models, Acadamic Press (Chapter 5.1 to 5.3, Chapter 6)
- C.W. Gardiner (2004). Handbook of Stochastic Methods. Springer (Chapter 3).
For a more advanced discussion of continuous-time Markov chains, see:
- S. Karlin and H.M. Taylor (1975), A First Course in Stochastic Processes. Academic Press (Chapter 4)
- E. Cinlar (1975). Introduction to Stochastic Processes. Prentice Hall (Chapter 8)
For a discussion of CTMC's with all the mathematical details included, see:
- L. Breiman (1968). Probability. Addison-Wellesley. (Chapter 15)
For a discussion of the use of CTMC models in chemistry (in which the CTMC models the reactions between different "species" of molecules), see:
- C.W. Gardiner (2004). Handbook of Stochastic Methods. Springer (Chapter 7).
- P. Whittle (1986). Systems in Stochastic Equilibrium. John Wiley (Chapter 7, Chapters 13, 14, 15, 16)
The latter book also makes clear the connection to Boltzmann processes and statistical thermodynamics.
Topic 6: Diffusions and Stochastic Differential Equations
A nice discussion of Brownian motion can be found in:
- S. Karlin and H.M. Taylor (1975), A First Course in Stochastic Processes. Academic Press (Chapter 7)
A deeper discussion is available in:
- L. Breiman (1968). Probability. Addison-Wellesley. (Chapter 12)
For a discussion of stochastic differential equations and diffusions, see:
- J. Goodman, K-S Moon, A. Szepessy, R. Tempone, and G. Zouraris. Stochastic and Partial Differential Equations with Adapted Numerics (available for download on course website, courtesy of R. Tempone).
- L. Breiman (1968). Probability. Addison-Wellesley. (Chapter 16).
- S. Karlin and H.M. Taylor (1981), A Second Course in Stochastic Processes. Academic Press (Chapter 15)
- C.W. Gardiner (2004). Handbook of Stochastic Methods. Springer (Chapter 4).
- J.M. Steele (2001). Stochastic Calculus and Financial Applications. Springer
- B. Oksendal (2003). Stochastic Differential Equations. Springer