PhD defence | Daniel Otero, Function-valued Mappings and SSIM-based Optimization in Imaging

Function-valued Mappings and SSIM-based Optimization in Imaging 

In image processing there are two important paradigms upon which imaging tools are developed: representation of images as either real or valued-vector functions, and using the Euclidian distance as either a metric or a norm for images. The former offers a good framework for applications since some type of images, e.g., colour and hyperspectral images, can be well represented by a vector-valued function. As for the latter, this is a convex and mathematically tractable metric that is frequently used in a variety of imaging tasks in which an optimization problem is to be solved (e.g., deblurring, denoising, sparse reconstruction, etc.).

Although many efficient imaging tools have been developed which are based on these paradigms, these approaches have some limitations. For instance, consider the case of a video sequence, which can be represented as a vector-valued function u(x) : Ω → Rⁿ. Here, Ω is the spatial domain, and each component u_i of the range of u, 1 ≤ i ≤ n, is a time frame. Due to the finite dimensionality of the range, it is not possible to propose a suitable space of functions as a model for the class of time functions defined at each pixel location of the video. A similar situation is encountered when complex data sets such as hyperspectral images are to be modelled.

As for the Euclidian distance, measures of visual quality such as the Mean Square Error (MSE) and Peak to Signal Noise Ratio (PSNR) are Euclidian-based metrics, however, it has been shown that these metrics are not the best choice when it comes to measure the visual quality of images. This drawback of Euclidian-based metrics has been overcome up to some point by the Structural Similarity Index Measure (SSIM), which has shown to be a good model of the Human Visual System (HSV) [3]. For this reason, the SSIM has been employed as a fidelity term in several types of optimization problems for carrying out a variety of imaging tasks—e.g., filtering, best-SSIM approximation, video coding, etc. [1, 2, 4]. Nevertheless, these methods tend to be developed for particular applications and are not general.

In this thesis we propose two alternative approaches to overcome the difficulties mentioned above, namely, Function-valued Mappings (FVMs) and SSIM-based Optimization. With FVMs we address the problem of modelling complex data sets properly. In this case, we define a FVM as follows:

u(x) : X → F(Y),

where both X and F(Y) are Banach spaces, and F(Y) is a space of real-valued functions whose support is the set Y . Observe that the range of u is infinite dimensional, therefore, depending on the application, an appropriate space F(Y) can be chosen to describe the functions that are assigned to each x ∈ X. For instance, a hyperspectral image can be represented as a FVM of the form u(x) : X ⊂R² → L(R)² is the space of square integrable functions supported on the real line—that is, we are interested in the spectral functions that have finite energy. Furthermore, based on the FVM approach, we introduce a Fourier transform and a new class of fractal transforms for FVMs.

Regarding SSIM-based optimization, we present a general framework for optimization problems that involve the SSIM as a fidelity term. Such a framework is divided into two approaches, namely, unconstrained and constrained SSIM-based optimization. In the unconstrained approach, an optimization problem is defined as follows:

min {T (Φ(x), y) + λ h(x)}, 

where T (·,·) := 1−SSIM(·,·) is a SSIM-based dissimilarity measure, Φ is a linear transformation, y a given observation, and h(x) a convex regularizing term along with its regularization parameter λ . As for the constrained counterpart, the minimization problem is given by

min T (Φ(x), y)

subject to h_i(x)≤0, i=1,…,m

Ax=b,

where the h_i(x) define a set of convex constraints, and Ax = b is an equality constraint. We also present a set of algorithms to solve both (2) and (3). The advantage of this framework is that it allows us to involve the SSIM in optimization problems in which it had not been employed before, in particular, SSIM optimization with norm regularization.

References

    [1]  D. Brunet, E.R. Vrscay and Z. Wang, “Structural similarity-based approximation of signals and images using orthogonal bases,” in ICIAR 2010, Vol. 6111 of Lecture Notes in Computer Science, Springer, 11–22 (2010).

    [2]  S.S. Channappayya, A.C. Bovik, C. Caramanis, R.W. Heath, Jr., “Design of linear equalizers optimized for the structural similarity index.” IEEE Transactions on Image Processing, 17(6), 857–872 (2008).

    [3]  Z. Wang, A.C. Bovik, H.-R. Sheikh and E.S. Simoncelli, “Image quality assessment: From error visibility to structural similarity.” IEEE Trans. Image Processing, 13(4), 600–612 (2004).

    [4] S.Wang,A.Rehman,Z.Wang,S.MaandW.Gao,“SSIM-motivated rate-distortion optimization for videocoding.” IEEE Trans. Circuits Syst. Video Techn., 22(4), 516–529 (2012).