Candidate
Mehrdad Jabbarzadeh Gangeh
Title
Kernelized Supervised dictionary Learning
Supervisors
Kamel, Mohamed S. and Ghodsi, Ali (Stats)
Abstract
The representation of a signal using a learned dictionary instead of predefined operators, such as wavelets, has led to the state-of-the-art results in various applications such as denoising, texture analysis, and face recognition. The area of dictionary learning is closely associated with sparse representation and this means that the signal is represented using few atoms in the dictionary. Despite of recent advances in the computation of dictionary using fast algorithms such as K-SVD, online learning, and cyclic coordinate descent, which make the computation of dictionary on millions of data samples computationally feasible, the dictionary is mainly computed using unsupervised approaches such as k-means. These approaches learn the dictionary by minimizing the reconstruction error without taking into account the category information, which is not optimal in classification tasks.
In this thesis, we propose a supervised dictionary learning (SDL) approach by incorporating information on class labels into the learning of dictionary. To this end, we propose to learn the dictionary in a space where the dependency between the signals and their corresponding labels is maximized. To maximize this dependency, the recently introduced Hilbert Schmidt independence criterion (HSIC) is used. The learned dictionary is compact and has closed form; the proposed approach is fast. We show that it outperforms other unsupervised and supervised dictionary learning approaches in the literature on real-world data.
Moreover, the proposed SDL approach has this main advantage that it can be easily kernelized by incorporating a kernel, particularly a data-derived kernel such as compression-based kernels, into the formulation. In this thesis, we propose a novel compression-based (dis)similarity measure. The proposed measure utilizes 2D MPEG-1 encoder, which takes into consideration the spatial locality and connectivity of pixels in the images. The proposed formulation has been carefully designed based on MPEG encoder functionality. To this end, by design, it solely uses P-frame coding to find the (dis)similarity among patches/images. We show that the proposed measure works properly on both small and large patch sizes on textures. Experimental results show that by incorporating the proposed measure as a kernel into our SDL, it significantly improves the performance of a supervised pixel-based texture classification on Brodatz and outdoor images compared to other compression-based dissimilarity measures as well as state-of-the-art SDL methods. It also improves the computation speed by about 40% compared to its closest rival.