Syamantak Datta Gupta
On Linear MMSE Approximations of Stationary Time Series
In a large number of applications arising in various fields of study, time series are approximated using linear MMSE estimates. Essentially, the time series of interest is estimated as a linear combination of the past values of the series itself, or other time series that influence the former. Such approximations include finite order moving average and autoregressive approximations as well as the causal Wiener filter. In this dissertation, we study two topics related to the estimation of wide sense stationary (WSS) time series using linear MMSE estimates. First, we study the convergence properties of finite order linear MMSE estimates of a WSS time series. Next, we study the problem of detecting causal connections within a family of WSS time series using linear MMSE estimates.
In the first part of this dissertation, we study the asymptotic behaviour of autoregressive (AR) and moving average (MA) approximations. Our objective is to investigate how faithfully such approximations replicate the original sequence, as the model order as well as the number of samples approach infinity. We consider two aspects: convergence of spectral density of MA and AR approximations when the covariances are known and when they are estimated. Under certain mild conditions on the spectral density and the covariance sequence, it is shown that the spectral densities of both approximations converge in L2 as the order of approximation increases. It is also shown that the spectral density of AR approximations converges at the origin under the same conditions. Under additional regularity assumptions, we show that similar results hold for approximations from empirical covariance estimates.
In the second part of this dissertation, we address the problem of detecting interdependence relations within a group of WSS time series. The objective is to understand the interaction of different time series and to determine whether one series is causally influenced by others. We use Granger-causality as a tool to identify and measure causal connections. Ideally, in order to infer the complete interdependence structure of a complex system, dynamic behaviour of all the processes involved should be considered simultaneously. However, for large systems, use of such a method may be infeasible and computationally intensive, and pairwise estimation techniques may be used to obtain sub-optimal results. In this dissertation, we investigate the problem of determining Granger-causality in an interdependent group of jointly WSS time series by using pairwise causal Wiener filters. Analytical results are presented, along with simulations that compare the performance of a method based on finite impulse response (FIR) Wiener filters to another using directed information, a tool widely used in literature. The problem is studied in the context of cyclostationary (CS) processes as well. Finally, a new technique is proposed that allows the determination of causal connections under certain sparsity conditions.