ECE 686 - Filtering and Control of Stochastic Linear Systems
Instructor:
Professor
Christopher
Nielsen
Office:
EIT
4106.
Office
hours:
By
appointment.
Contact:
cnielsen@uwaterloo.ca
;
telephone
extension
32241.
Website:
http://learn.uwaterloo.ca/
Location and time
EIT 3141, Tuesdays, 11:30 am – 2:20 pm
Course Description
This course is concerned with discrete-time systems subject to disturbances. We seek to estimate quantities associated to these systems and to optimally control the evolution of these systems. Broadly speaking, we are interested in decision making under uncertainty.
The first half of the course establishes the fundamentals of the estimation problem, culminating in the derivation of the fact that state estimation in linear systems is equivalent to projection onto a closed linear subspace generated by an observation process in a Hilbert space of random variables. This leads to the Kalman filter, which finds use in many applications ranging from aerospace to finance. The course will then cover control and estimation for input-output models and the issues of stochastic optimal control (based on dynamic programming), the linear quadratic Gaussian (L.Q.G.) control problem and optimal control of Markov chains over infinite horizons.
Recommended background
Linear Algebra (MATH215), Probability (ECE316), Multivariable Control Systems (ECE682), Stochastic Processes (ECE604), or consent of instructor.
Required text
There is no required text for this course. Instructor will write notes on the black board. The course mostly follows the textbook Stochastic Systems: Estimation, Identification and Adaptive Control, P. R. Kumar and P. Varaiya.
Additional references
- Stochastic Processes, Estimation and Control, J. L. Speyer and W. H. Chung.
- Stochastic Modelling and Control, M. Davis and R. Vinter.
- Dynamic Programming and Optimal Control, D. Bertsekas.
- Principles of Mathematical Analysis, W. Rudin.
Evaluation
50%
Final
exam
:
open
book.
20%
Midterm
exam
:
open
book.
30%
Assignments
:
3
or
4
assignments
posted
over
the
course
of
the
term.
Rules for group work in assignments: You are expected to work on the problem sets on your own. You are encouraged to discuss problems with classmates, but you will be penalized by splitting marks if assignment solutions do not show independent thought.
Tentative Topics List
-
Introduction
- Stochastic state models, examples.
-
Mathematical
preliminaries
- Infinite sets, normed vector spaces, inner product spaces, Hilbert spaces and the projection theorem.
-
Measure
theoretic
probability
theory
- Systems of sets, measure spaces, random variables, expected values, the space of square integrable random variables.
-
Classical
estimation
theory
- Minimum mean square error estimator, linear least squares estimator, affine least squares estimator, least squares parameter estimation.
-
Recursive
estimation
and
Kalman
filtering
- Recursive linear estimation, discrete-time Kalman filter, linear time-invariant systems.
-
Input-output
models
- ARMAX models, prediction theory for ARMAX systems, minimum variance control of ARMAX systems.
-
Stochastic
optimal
control
and
dynamic
programming
- Deterministic optimal control, linear quadratic regulation, stochastic optimal control with complete observations, linear quadratic Gaussian control, LQG with partial observations.
-
Control
of
Markov
chains
over
infinite
horizons
- homogenous Markov chains, Perron-Frobenius theory of non-negative matrices, discounted cost criterion, algorithms for computing optimal policies.
Required inclusions
- Academic integrity: In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility.
- Grievance: A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70, Student Petitions and Grievances, Section 4. When in doubt please be certain to contact the department’s administrative assistant who will provide further assistance.
- Discipline: A student is expected to know what constitutes academic integrity to avoid committing an academic offence, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) or about “rules” for group work/collaboration should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean. For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline. For typical penalties check Guidelines for the Assessment of Penalties.
- Appeals: A decision made or penalty imposed under Policy 70 (Student Petitions and Grievances) (other than a petition) or Policy 71 (Student Discipline) may be appealed if there is a ground. A student who believes he/she has a ground for an appeal should refer to Policy 72 (Student Appeals).
- Note for students with disabilities: The AccessAbility Services, located in Needles Hall, Room 1132, collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with the AccessAbility Services at the beginning of each academic term.