Perhaps the greatest contribution from the math center was the creation of Information Theory
[27] by Claude Shannon in 1948. For perhaps the first 25 years of its existence, Information
Theory was regarded as a beautiful theory but not as a central guide to the architecture and
design of communication systems. After that time, however, both the device technology and the
engineering understanding of the theory were sufficient to enable system development to follow
information theoretic principles.
A number of information theoretic ideas and how they affect communication system design will
be explained carefully in subsequent chapters. One pair of ideas, however, is central to almost
every topic. The first is to view all communication sources, e.g., speech waveforms, image
waveforms, and text files, as being representable by binary sequences. The second is to design