I have only begun preparing this course site.

Below is the material I plan to cover in EE378 this upcoming Spring quarter.
The course title as it currently appears in various listings is "Estimation
and Detection", but  I have almost entirely neglected the
"Detection" part. Probably a more suitable name for the course would have
been "Introduction to Statistical Signal Processing" if this title had not
been already taken by its prerequisite EE278.
Focus will be on the basics of discrete-time random signal estimation,
prediction, filtering, parameter estimation, and spectrum estimation.
In the choice of material I have attempted to minimally overlap with other
courses in our department (in particular the grad level comm. courses, which
is why I neglected the Detection part), as well as with stat courses that
thoroughly cover various aspects of estimation theory.


This course is the natural continuation to EE278.

1.      Basic Concepts

1.1.   Gaussian Processes

1.2.   Stationarity (weak and strict) and (Mean-Square) Ergodicty

1.3.   Slutsky's Theorem

1.4.   Spectral distribution and density



2.      Hilbert space representation of stationary random signals



3.      Regular Processes

3.1.   Minimum phase systems

3.2.   Regularity

3.3.   Spectral Factorization

3.4.   ARMA processes

3.5.   State-space models



4.      A bit on Sampling of Continuous-time Band-limited  Random Signals



5.      Optimal Linear Estimation of Finite-Dimensional Vectors

5.1.   Orthogonality principle

5.2.   Cholesky's theorem

5.3.   Causal Filter



6.      Linear Estimation of a Random Signal

6.1.   Optimal linear estimation

6.2.   Linear prediction

6.3.   Wiener-Hopf equations and their solution



7.      Kolmogorov-Szego Formula



8.      Wold Decomposition

8.1.   Predictable (aka 'deterministic') and weakly predictable processes

8.2.   Decomposition of a general WSS process



9.      Optimal Linear Estimation: Causal- and Non-Causal Wiener Filtering

9.1.   Non-causal Wiener filter (reminder from EE278)

9.2.   Causal Wiener filter



10.  Optimal Linear Estimation in the State-Space: Kalman Filtering



11.  Parameter Estimation

11.1.                    Bias and variance of an estimator

11.2.                    UMVU estimators, James-Stein estimator

11.3.                    Fisher information

11.4.                    Cramer-Rao Bound (including vector case and case of
biased estimators)

11.5.                    Improved version of CR: Chapman-Robbins bound

11.6.                    Asymptotic consistency and asymptotic efficiency

11.7.                    Maximum-Likelihood Estimation

11.8.                    The method of moments

11.9.                    Least Squares estimation



12.  Spectrum Estimation

12.1.                    A-parametric spectrum estimation

12.2.                    Bartlett's formula

12.3.                    The periodogram estimator

12.4.                    Periodogram averaging, periodogram windowing


According to remaining time and interest, may try to touch some of the more
modern material, e.g.:


- Hidden Markov process state estimation

- The EM algorithm

 - Particle Filtering

Bibliography:

Primary books:

-         A. Papoulis, Probability, Random Variables and Stochastic Processes, 3rd Ed. , McGraw-Hill, 1991

-         B. Porat, Digital Processing of Random Signals, Prentice-Hall, 1994

-         Brockwell and Davis, Introduction to Time Series and Forecasting, Springer 1996

-         T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall, NJ, ISBN 0-13-022464-2, 854pp, 2000

-         J. L Doob, Stochastic Processes, John Wiley & Sons (1953)

- L. D. Davisson and R. M. Gray  Introduction to Statistical Signal Processing, Cambridge University Press, 2004.

Auxiliary or Peripherally Related Texts:

-    E. L. Lehmann, Theory of Point Estimation, Springer, 2nd ed. 1998.

-    B.Porat, A Course in Digital Signal Processing, John Wiley and Sons, 1997

-    T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley, 1991

-         W. Rudin, Real and Complex Analysis, Third Edition, McGraw-Hill, Inc., 1987

-    Petersen, K. Ergodic Theory. Cambridge, England: Cambridge University Press, 1983

-          R. M. Gray, Entropy and Information Theory, revised November 2000

Course Requirements