One of the most reliable sources for the diagnosis of different types of cancers is the histopathological examination of biopsy samples. Thanks to the improvements of the computer-aided diagnosis systems in recent years, we can get considerable assistance from computers for the classification of malignancies. Medical experts can take advantage of novel image classifier and search-based programs for a tangible interpretation of the histopathological images.
Several supervised classification methods are proposed in medical imaging literature. Many of these methods can result in high accuracy levels. However, typically they are not able to provide expert-friendly justification. On the contrary, unsupervised search-based methods can provide justifications for their final decision. These methods search in order to find the most similar images to a query image and classify it using majority voting among retrieved images along with their corresponding metadata. As a result, with the provision of the metadata of the retrieved cases, medical specialists are able to justify the final decision in the search-based classification methods. Although, the search-based methods can provide justification for their classification results, most of the times their accuracy is lower than the supervised classification techniques. Therefore, in this project, we combine the search-based methods with supervised classification and propose a class-aware search to increase the accuracy of the content-based search. Our method is designed to operate on deep image embeddings with a wide applicability to different histopathological datasets.
We investigate two different scenarios. In one scenario, we apply the supervised classification to the query image and then use its results to restrict the image search to dominant labels rather than all labels. In the second scenario, we modify the weighted search algorithm by class-aware adjustment of matching distances. These scenarios are implemented on two publicly available datasets containing endometrial and colorectal cancer images and the results show that both proposed scenarios can enhance the accuracy of search-based classification.
Figure 1-Sample patches from the datasets we used to validate proposed scenarios. (A): Endometrial cancer dataset and (B): Colorectal cancer dataset.
Figure 2-Illustration of the proposed scenarios. In scenario A, we apply the supervised classification to the query image and then use its results to restrict the image search to dominant labels rather than all labels. In scenario B, we modify the weighted search algorithm by class-aware adjustment of matching distances.
Ref: A. Ebrahimian, H. Mohammadi, M. Babaie, N. Maftoon and H. R. Tizhoosh, "Class-Aware Image Search for Interpretable Cancer Identification," in IEEE Access, vol. 8, pp. 197352-197362, 2020, doi: 10.1109/ACCESS.2020.3033492.