EE 276: Readings

Stanford University, Tsachy Weissman, Winter Quarter 2023-24

Books

  • T. M. Cover and J. A. Thomas, Elements of Information Theory.

  • M. M. Wilde, From Classical to Quantum Shannon Theory. (Chapter 2 contains a nice summary of classical information theory)

  • Soni, Jimmy, and Rob Goodman. A mind at play: How Claude Shannon invented the information age. Simon and Schuster, 2017. [Amazon] (Biography of Claude Shannon)

  • MacKay, David JC. Information theory, inference and learning algorithms. Cambridge university press, 2003. [online] (Really cool book on information theory and learning with lots of illustrations and applications)

Papers

  • Shannon, Claude E. “Prediction and entropy of printed English.” Bell system technical journal 30.1 (1951): 50-64. [pdf] (Shannon's attempt at estimating the entropy of English by experimenting with human compressors).

  • Shannon, Claude E. “A mathematical theory of communication.” Bell system technical journal, 27.3 (1948), 379-423. [pdf] (Shannon's original paper that gave birth to information theory).

Videos

Fun