ECE 686 - Filtering and Control of Stochastic Linear Systems
Instructor:
Professor
Stephen
L.
Smith
Office:
E5
5112.
Email:
stephen.smith@uwaterloo.ca
Please put "ECE686" at the front of your subject line for all course related email.
Lectures
EIT 3141, Thursdays, 11:30 am – 2:20 pm. Lectures will not be held during reading week (February 20 - 24).
Course Motivation
Broadly speaking, this is a course on decision making under uncertainty. Specifically, the course focuses on the study of stochastic linear systems where the dynamics have certain components of a stochastic nature. The course will cover both estimation and control of such systems. The first half of the course establishes the fundamentals of the estimation problem, culminating in the derivation of the fact that state estimation in linear systems is equivalent to projection onto a closed linear subspace generated by an observation process in a Hilbert space of random variables. This leads to the Kalman filter, which finds use in many applications ranging from aerospace to finance. The course will then cover the issues of stochastic optimal control (based on dynamic programming), control of Markov chains, and optimal stopping time problems (e.g., when is the optimal time to buy or sell a product?).
Course outline
The following is tentative list of topics that will be covered in the course.
-
Introduction
and
Mathematical
Background
- Motivation and overview
- Review of probability theory and stochastic processes
-
Derivation
of
the
Projection
Theorem
- Metric and linear spaces
- Hilbert spaces and the projection theorem for linear estimation
-
Estimation
of
Discrete-Time
Linear
Systems
- Least-squares and recursive estimation
- The discrete-time Kalman filter
-
Stochastic
Optimal
Control
- Dynamic programming and the principle of optimality
- Optimal control with complete and partial observations
- Linear quadratic regulators
-
Control
of
Markov
Chains
- Overview of Markov chains
- Markov policies and the cost of optimal policies
- Optimal Stopping Time Problems
Textbook
There is no required textbook. Parts of the course are based on notes by Professor Andrew Heunis, and the following book:
- P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control, 1986.
The following textbooks may also be useful for additional information:
- J. L. Speyer and W. H. Chung, Stochastic Processes, Estimation and Control, 2008
- D. P. Bertsekas, Dynamic Programming and Optimal Control, 2005
Grading
The course will consist of assignments, a midterm, and a final exam. There will be a total of four homework assignments, given roughly once every three weeks. The grading scheme is
- Assignments: 20%
- Midterm: 30%
- Final Exam: 50%
General University of Waterloo guidelines
- Academic integrity: In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility.
- Grievance: A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70, Student Petitions and Grievances, Section 4. When in doubt please be certain to contact the department’s administrative assistant who will provide further assistance.
- Discipline: A student is expected to know what constitutes academic integrity to avoid committing an academic offence, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) or about “rules” for group work/collaboration should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean. For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline. For typical penalties check Guidelines for the Assessment of Penalties.
- Appeals: A decision made or penalty imposed under Policy 70 (Student Petitions and Grievances) (other than a petition) or Policy 71 (Student Discipline) may be appealed if there is a ground. A student who believes he/she has a ground for an appeal should refer to Policy 72 (Student Appeals).
- Note for students with disabilities: The AccessAbility Services, located in Needles Hall, Room 1132, collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with the AccessAbility Services at the beginning of each academic term.