Logical Representational and Computational Methods for Markov Decision Processes
Craig Boutilier
Department of Computer Science
University of Toronto
Toronto, Ontario, Canada M5S 3H5
(cebly@cs.toronto.edu)
http://www.cs.toronto.edu/~cebly
Prerequisites: This course presumes a familiarity (or comfort level) with
elementary (discrete) probability theory and inference, and propositional
and predicate logic. Some familiarity with Bayesian networks will be helpful,
but will not be assumed.
Summary: Markov decision processes (MDPs) have become standard models for
sequential decision problems involving uncertainty within the planning
and probabilistic reasoning communities. The aim of the course is:
(a) to provide an introduction to Markov decision processes;
(b) to survey some of the recent advances that have been made in the
concise and natural representation of MDPs using logical techniques; and
(c) to survey some of the computational methods for solving MDPs that
exploit this logical structure.
Representations to be discussed include propositional methods (e.g.,
probabilistic STRIPS, dynamic Bayesian networks), first-order representations,
representations using temporal logics, and computationally-motivated
representations drawn from the verification community (e.g., BDDs).
Computational techniques include regression, abstraction, and decomposition
methods that rely on these specific logical representations.
Course Notes:
Lecture 1
Lecture 2
Lecture 3
Lecture 4
Lecture 5
Back to course listing.