EE 376A : Information Theory

Announcements

• Final grades have been posted on CCNET.

• Graded finals are ready for pickup from the EE376A drawer on the second floor of the Packard Building.

• Final solution has been posted.

• Final exam will be on Monday, March 18, 3:30-6:30 PM, at 370-370.

• There will be an additional office hour on Sunday, Mar 17, 1:30PM-2:30PM, at Packard 214

• Homework 8 solution has been posted.

• The video of Review Session has been uploaded. Here are the Notes for the review session [Note 1] [Note 2]

• The video of Lecture 20 has been uploaded.

• Sample Final and solution have been posted.

• Homework 8 (due on 3/14) has been posted.

• Graded midterms are ready for pickup from the EE376A drawer on the second floor of the Packard Building.

• Midterm grades have been posted on CCNET.

• Midterm solution has been posted.

• Sample Midterm and solution have been posted.

• We have set up a forum in piazza where you can post and discuss about the homeworks.

Course Overview

Information theory is the science of operations on data such as compression, storage, and communication. It is among the few disciplines fortunate to have a precise date of birth: 1948, with the publication of Claude E. Shannon's paper entitled "A Mathematical Theory of Communication".

Our course will explore the basic concepts of Information theory. It is a prerequisite for research in this area, and highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal proces sing. The intimate acquaintance that we will gain with measures of information and uncertainty - such as mutual information, entropy, and relative entropy - would be invaluable also for students, researchers, and practitioners in fields ranging from neuroscience to machine learning. Also encouraged to enroll are students of statistics and probability, who will gain an appreciation for the interplay between information theory, combinatorics, probability, and statistics.