Pratheepa Jegannathan
McMaster University
Room: M3 3127
A Transformer-based Framework for Sparse Clustering
Sparse clustering algorithms traditionally identify cluster partitions and a global feature weight vector by solving a constrained optimization problem.
However, the global $L_{1}$ penalties used in existing sparse clustering frameworks often fail to capture the heterogeneous features that drive different clusters.
In this talk, I will first review sparse clustering algorithms formulated as constrained optimization problems and discuss the iterative estimation procedures commonly used to obtain their solutions. I will then introduce a Transformer-based framework that enables cluster-specific feature selection within a sparse clustering algorithm. The proposed approach leverages attention heads to learn similarities between observations and uses feed-forward networks as feature-selection gates while incorporating a constrained objective function. We estimate the model using a stage-wise optimization procedure, which enables prototype learning and the identification of cluster-specific features.
I will conclude by presenting experimental results demonstrating that the proposed Transformer-based framework identifies cluster-specific features across different simulation settings.