Candidate
Adam Daniel Gomes
Title
Prediction for Projection on Time-varying Surfaces
Supervisor
David Wang
Abstract
In spatial augmented reality applications, when video projectors display images on time- varying, non-planar surfaces, rather than on flat, rigid surfaces, undesired image distortion may occur. For applications where realism is of the utmost importance, such as surgical simulations, image distortion can significantly detract from the user experience. To combat this, the time-varying surface can be modelled using a mass-spring model, commonly used for simulating deformable objects in computer graphics. The mass-spring model can be formulated into a non-linear state space equation which describes the dynamics of discrete points making up the surface of the object. Two simulation techniques are used to verify the model and to determine the best approach for real-time simulations.
To project images in real-time onto quickly changing surfaces, an extended Kalman filter (EKF) prediction algorithm is developed to predict the position of the deforming surface, at a specified point in time, T_s seconds, in the future. Using the linearized mass-spring system, the EKF is formulated and tested upon two simulation scenarios. The simulation scenarios include a falling cloth with added process noise, and a cloth perturbed by random viscous forces. Using mean squared error, the results show the EKF predictions and simulation outputs are nearly convergent. For each scenario, the parameters of the EKF are manually tuned to improve the accuracy of the predictions. Experimental data is collected by measuring the movement of cloth-like materials to verify the effectiveness of the prediction algorithm. Specifically, cloth movement data is captured using infra-red markers and motion capture software. The EKF prediction algorithm is run on the experimental data producing near convergent results between the predictions and the measurements.
When the physical surface is changing noticeably and quickly, compared to the projector’s drawing rate, additional distortion may occur. An inter-frame prediction algorithm is developed to further predict the position of discrete points at their corresponding projection times, when the frame rate is faster than the EKF.
When implementing the EKF in real-time, there is a trade-off between speed and accuracy. If the number of discrete points is large, the EKF is required to solve a large system of equations. To combat this, non-linear optimization techniques are used to find parameters that reduce the number of states while maintaining system dynamics. This results in a sparser, more computationally efficient model with similar physical behaviour to the original system.
Applications for time-varying surface prediction include surgical simulations, projection for entertainment and advertising, and other spatial augmented reality applications.