Deep neural networks can learn deep feature representation for hyperspectral image (HSI) interpretation and achieve high classification accuracy in different datasets. However, counterintuitively, the classification performance of deep learning models degrades as their depth increases. Therefore, we add identity mappings to convolutional neural networks for every two convolutional layers to build deep residual networks (ResNets). To study the influence of deep learning model size on HSI classification accuracy, this paper applied ResNets and CNNs with different depth and width using two challenging datasets. Moreover, we tested the effectiveness of batch normalization as a regularization method with different model settings. The experimental results demonstrate that ResNets mitigate the declining-accuracy effect and achieved promising classification performance with 10% and 5% training sample percentages for the University of Pavia and Indian Pines datasets, respectively. In addition, t-Distributed Stochastic Neighbor Embedding (t-SNE) provides a direct view of the extracted features through dimensionality reduction.