ECE 612 - Winter 2015

ECE 612 - Information Theory

Instructor

Professor Amir Khandani

Description

The objective of this course is to explain the fundamental concepts of information theory focusing on problems in communications (efficient signal representation and reliable transmission). Specifically, we will discuss topics in source/channel coding, and networking. Along with a theoretical approach, a goal of the course is to clarify the practical issues behind the corresponding theory.

Some specific topics to be covered are as follows:

  1. Introduction:
    1. Basic definitions
    2. Asymptotic Equipartition Property (AEP)
    3. Analysis of Markovian sources and related coding issues
  2. Coding for source representation:
    1. Constrained coding for noiseless channels
    2. Huffman coding
  3. Channel capacity, Random coding, Source/Channel coding theorems, Gaussian channel.
  4. Network information theory:
    1. Multiple access channel
    2. Broadcast channel
  5. Rate distortion theory.

References

T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, New York, 1991.

R. G. Gallager, Information Theory and Reliable Communication, Wiley, New York 1968.

J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering, Wiley, 1965.

R. E. Blahut, Principles and Practice of Information Theory, Addison Wesley, 1987.

Also see the course website.

Course evaluation

A set of assignments, mid-term and final.