Books
Sections of the following books may be useful in parts of the course.
T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall, NJ, 2000.
J. M. Bernardo and A. F. M. Smith, Bayesian Theory, Wiley, 2000.
E. L. Lehmann, Theory of Point Estimation, Springer, 2nd ed. 1998.
T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley, 1991.
D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, UK, 2003.
P. D. Grunwald, The Minimum Description Length Principle, MIT Press, June 2007.
N. Cesa-Bianchi, and G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, New York, 2006.
The following books may be conferred for further reading on some of the topics we will touch.
Classical statistical signal processing and time series analysis
B. Porat, Digital Processing of Random Signals, Prentice-Hall, 1994.
A. Papoulis, Probability, Random Variables and Stochastic Processes, 3rd ed., McGraw-Hill, 1991.
Brockwell and Davis, Time series: theory and methods, 2nd ed., Springer 1991.
J. L Doob, Stochastic Processes, John Wiley & Sons, 1953.
L. D. Davisson and R. M. Gray, Introduction to Statistical Signal Processing, Cambridge University Press, 2009.
W. A. Gardner, Introduction to Random Processes: with Applications to Signals and Systems, 2nd ed., McGraw-Hill, 1990.
H. Stark and J. W. Woods, Probability and Random Processes, with Applications to Signal Processing, Prentice-Hall, 3rd ed., 2001.
C. W. Therrien, Discrete Random Signals and Statistical Signal Processing, Prentice-Hall, 1992.
H. L. Van Trees, Detection, Estimation and Modulation Theory, Part I, Wiley, 1968.
Related to more modern signal processing
A. Doucet, N. de Freitas, and N. J. Gordon, eds., Sequential Monte Carlo Methods in Practice, Springer, New York, 2001.
F. V. Jensen and T. D. Nielsen, Bayesian Networks and Decision Graphs, 2nd ed., Springer, 2007.
L. Lauritzen, Graphical Models, Clarendon Press, Oxford, UK, 1996.
B. Ristic, S. Arulampalam, and N. Gordon, Beyond the Kalman Filter: Particle Filters for Tracking Applications, Artech House, Boston, 2004.
S. Thrun, W. Burgard and D. Fox, Probabilistic Robotics, MIT Press, 2005.
R. G. Cowell, A. P. Dawid, S. L. Lauritzen, and D. J. Spiegelhalter, Probabilistic Networks and Expert Systems, Springer, New York, 1999.
D. Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques, MIT Press, 2009.
Xavier Guyon, Random fields on a network: Modeling, Statistics and Applications, Probability and its Applications, Springer-Verlag, New York, 1995.
Miscellaneous
Luc Devroye, Gabor Lugosi and Laszlo Gyorfi, A Probabilistic Theory of Pattern Recognition, Springer-Verlag, New York, 1996.
David Williams, Probability with Martingales, Cambridge Mathematical Textbooks, 1st ed. 2001.
K. Petersen, Ergodic Theory, Cambridge, England: Cambridge University Press, 1983.
R. M. Gray, Entropy and Information Theory, revised 2009.
Luc Devroye and Gabor Lugosi, Combinatorial Methods in Density Estimation, New York: Springer, 2001.
P. Bremaud, Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queue,
Springer, New York, 1999.
R. M. Gray, Toeplitz and Circulant Matrices: A Review, revised 2006.
M. Mezard and A. Montanari, Information, Physics, and Computation, Oxford University Press, Inc., New York, 2009.
Papers
Hidden Markov models
Y. Eprahim and N. Merhav, “Hidden Markov Processes,” IEEE Trans. on IT, vol. 48, vol. 6, Jun. 2002.
G. D. Forney, Jr., “The Viterbi Algorithm,” Proc. IEEE, vol. 61, no. 3, pp. 268-278, Mar. 1973.
L. R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
Graphical models
B. J. Frey and N. Jojic, “A Comparison of Algorithms for Inference and Learning in Probabilistic Graphical Models,” IEEE Trans. Pattern Anal. Machine Intell., vol. 27, no. 9, pp. 1392-1416, Sept. 2005.
M. I. Jordan, “Graphical Models,” Statist. Sci., vol. 19, no. 1, pp. 140-155, 2004.
F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor Graphs and the Sum-Product Algorithm,” IEEE Trans. IT, vol. 47, no. 2, pp. 498-519, Feb. 2001.
H.-A. Loeliger, “An Introduction to Factor Graphs,” IEEE Signal Processing Mag., pp. 28-41, Jan. 2004.
H.-A. Loeliger, J. Dauwels, J. Hu, S. Korl, L. Ping, and F. R. Kschischang, “The Factor Graph Approach to Model-Based Signal Processing,” Proc. IEEE, vol. 95, no. 6, pp. 1295-1322, June 2007.
J. S. Yedidia, W. T. Freeman, and Y. Weiss, “Understanding Belief Propagation and its Generalizations,” Exploring Artificial Intelligence in the New Millenium, Science and Technology Books, Jan. 2003.
Particle and approximate non-linear Filtering
S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, “A Tutorial on Particle Filters for On-line Non-linear/Non-Gaussian Bayesian Tracking,” IEEE Trans. on SP, vol. 50, 2001.
D. Brigo, B. Hanzon, F. Le Gland. “Approximate Nonlinear Filtering by Projection on Exponential Manifolds of Densities,” Bernoulli, vol. 5, no. 3 (1999), pp. 495-534.
MDL
A. Barron, J. Rissanen, and B. Yu (1998), “The Minimum Description Length principle in coding and modeling,” IEEE. Trans. on IT, Oct., pp. 2743-2760.
N. Merhav and M. Feder, “Universal prediction,” IEEE Trans. on IT, vol. 44, no. 6, pp. 2124-2147, Oct. 1998.
J. Rissanen, Stochastic complexity and modeling, The Annals of Statistics, 1986.
J. Rissanen, Stochastic Complexity in Statistical Inquiry, World Scientifc Publishing Company, 1989.
J. Rissanen, “Fisher information and stochastic complexity,” IEEE Trans. on IT, vol. 42 no. 1, pp. 40-47, 1996.
N. Merhav and M. Feder, “A strong version of the redundancy-capacity theorem of universal coding,” IEEE Trans. on IT, vol. 41, no. 3, pp. 714-722, May 1995.
J. Rissanen, “Universal coding, information, prediction, and estimation,” IEEE Trans. on IT, vol. 30, pp. 629-636, July 1984.
J. Rissanen, “Complexity of strings in the class of Markov sources,” IEEE Trans. on IT, vol. 32, pp. 526-532, 1986.
B. Clarke and A. R. Barron, “Information theoretic asymptotics of Bayes methods,” IEEE Trans. on IT, vol. 38, pp.453-471, 1990.
Information and estimation
D. Guo, S. Shamai, and S. Verdu, “Mutual Information and Minimum Mean-Square Error in Gaussian Channels,” IEEE Trans. on IT, vol. 51, no. 4, pp. 1261-1283, Apr. 2005.
S. Verdu, “Mismatched Estimation and Relative Entropy”.
S. Verdu, “Mismatched Estimation and Relative Entropy,” IEEE ISIT 2009, Seoul, Korea, June 28-July 3, 2009.
T. Weissman, “The Relationship Between Causal and Non-Causal Mismatched Estimation in Continuous-Time AWGN Channels”, preprint.
T. E. Duncan, “On the calculation of mutual information,” SIAM J. Appl. Math., vol. 19, pp. 215-220, July 1970.
A. R. Barron, “Entropy and the central limit theorem,” Annals of Probability, Vol.14, pp. 336-342, 1986.
Prediction, filtering, denoising of individual sequences
T. Weissman, E. Ordentlich, G. Seroussi, S. Verdu and M. J. Weinberger, “Universal discrete denoising: known channel,” IEEE Trans. on IT, vol. 51(1), Jan 2005.
E. Ordentlich, G. Seroussi, S. Verdu, M. Weinberger and T. Weissman, “Reflections on the DUDE,” IEEE Information Theory Society Newsletter, vol. 57, no. 2, pp. 5-10, June 2007 (invited).
T. Weissman, E. Ordentlich, M. Weinberger, A. Somekh-Baruch and N. Merhav, “Universal Filtering via Prediction,” IEEE Trans. on IT, vol. 53, no. 4, pp. 1253 - 1264, April 2007.
T. Moon and T. Weissman, “Universal FIR MMSE filtering”, IEEE Trans. on SP, vol. 57, no. 3, March 2009.
N. Merhav and M. Feder, “Universal prediction,” IEEE Trans. on IT, vol. 44, no. 6, pp. 2124-2147, Oct. 1998.
Some oldies
L. D. Brown, “Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems,” Annals of Mathematical Statistics, vol. 42, no. 3, pp. 855-903, June 1971.
A. Papoulis, “Predictable Processes and Wold's Decomposition: A Review,” IEEE Trans. on ASSP, vol. 33, no. 4, Aug 1985.
R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” Trans. ASME-J. Basic Eng., vol. 82, series D, pp. 35-45, 1960.
S. Geman and D. Geman (1984). “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 6, pp. 721-741, 1984.
|