ECE 710 - Estimation and Detection Theory
Instructor
Professor
Ravi
Mazumdar
Office:
EIT
3141
Phone:
(519)
888-4567
x37444
e-mail:
mazum@uwaterloo.ca
Lectures
Wednesdays and Thursdays, 10:00am to 11:20am in room EIT 3141
Course website
Pre-requisite
ECE604 or equivalent (Introduction to probability and random variables)
Course description
The principal aim of this course is to introduce the students to estimation and detection theory as a fundamental basis for dealing with random systems and signals. The course is intended for graduate students with interests in communications and networking, control systems, and signal processing. A course will present the salient ideas and tools associated with signal estimation and statistical detection theory and will emphasize the underlying structures.
The first part of the course will cover basic ideas from statistical point estimation theory introducing the concepts of efficiency and consistency of estimators. This will include minimum mean square estimates and maximum likelihood estimates and the role of the Fisher information as well as the Cramer-Rao lower bound. Then the course will cover the basics of Bayesian decision theory as applied to hypothesis testing. In particular the ideas of Likelihood Ratios or Radon-Nikodym derivatives and the Neyman-Pearson theory as well as Bayesian decision theory will be studied. This will conclude with the introduction of Kullback-Liebler distance and its implications.
The second part will deal with estimation theory for random sequences and signals. The first part will deal with wide sense stationary processes and the structure of Hilbert spaces associated with them culminating in the Wold decomposition. We will then study the problems of filtering, smoothing and prediction of stationary signals in the presence of noise. From stationary signals we will move on to non-stationary signals generated by noisy state-space models and the important ideas of Kalman filtering. We will study the asymptotic theory in detail. We will conclude with a discussion of parameter estimation for linear dynamic models.
The overall aim is to provide the student with a fundamental understanding of the structures involved in estimation and detection along with the underlying algorithmic and probabilistic ideas. At the end of the course the student will be well equipped to apply the ideas to the design of statistical signal processing algorithms, develop signal and parameter estimation algorithms, and formulate and solve statistical decision problems arising in communication and imaging problems.
Course outline
- Review of Probability: Expectations, Conditioning, the Gaussian distribution, CLT, and SLLN.
- Introduction to point estimation theory. Minimum mean square estimates.
- Consistency, efficiency of estimators. Fisher information matrix and the Cramer-Rao lower bound. Sufficient statistics and the Rao-Blackwell theorem.
- Introduction to hypothesis testing and detection theory. Likelihood ratios and the Radon-Nikodym theorem. Bayesian decision theory, The Neyman-Pearson lemma. Kullback-Liebler information.
- Introduction to wide sense stationary processes and their associated Hilbert spaces. The Wold decomposition. Filtering, prediction and smoothing of w.s.s. sequences.
-
Gauss-Markov
processes
and
state
space
models.
Discrete-time
Kalman
ltering.
Asymptotic
theory. - Continuous-time theory. Wiener and Kalman ltering.
Text and references
There is no single text that covers all the material adequately. Class notes on selected topics will be given out.
The following references are excellent for selected topics.
References
E.
Wong;
Introduction
to
random
processes,
Springer-Verlag,
1983.
B.
Picinbono,
Random
Signals
and
Systems,
Prentice-Hall,
1993.
T.
Kailath,
A.
Sayed,
and
B.
Hassibi;
Linear
Estimation,
Prentice-Hall,
1999.
V.
Poor;
Introduction
to
signal
detection
and
estimation,
Springer-Verlag.
A.
V.
Balakrishnan,
Kalman
ltering
theory,
Optimization
Software
Publ,
1995.
Course evaluation
- Weekly problem sets will be handed out. The onus is on you all to attempt them. Solutions will be posted.
-
There will be two in-term examinations. The dates will be announced later.
-
In addition, there will be one take home nal examination.
-
Marks distribution: Midterm= 40% , Final Exam = 60%
Additional remarks
- All in term exams will be closed book. You will be allowed to bring in one page of summary.
- If you miss the midterm exam no make-up exam will be given. If you have a valid reason then your final grade will be based on your performance in the rest of the course.
- Dishonesty will be dealt with harshly.