Cell-Level GNN-Based Prediction of Tumor Regression Grade in Colorectal Liver Metastases From Histopathology Images

Colorectal liver metastases (CLM) develop in almost half of patients with colon cancer. Response to systemic chemotherapy is the main determinant of patient survival. Due to the importance of assessing treatment response of CLM to chemotherapy for patient prognosis in the early stages, there is a need to classify tumor response grade (TRG) based on whole slide images (WSI). However, annotating WSI for training neural networks is a time-consuming task and inter-observer variability among pathologists presents a significant challenge in the accurate classification of TRG. In this work, we introduce an end-to-end pipeline for the prediction of TRG in CLM. The pipeline begins with color normalization of WSI via a Generative Adversarial Network (GAN), followed by precise nuclei segmentation combining a GAN with a Vision Transformer (ViT). The segmented nuclei are then clustered using a Graph Neural Network (GNN) to map the intricate cellular structures and interactions. Finally, TRG prediction is achieved through a Support Vector Machine (SVM) that integrates GNN outputs and clinical data, demonstrating enhanced accuracy compared to existing models. The proposed method achieves encouraging results at all stages. The GAN-based normalization and ViT-enhanced segmentation of WSIs yielded a high average Dice score of 82.5%. The GNN clustering of nuclei demonstrated precision, as evidenced by low Davies-Bouldin Index (DBI) scores, particularly for immune cells (0.40), cancer cells (0.45), and fibroblasts (0.42). Most notably, in TRG prediction, our model is able to stratify patients into two TRG classes (1-2 vs 3-5) with an accuracy of 91.3%.