Hybrid Multitask QSAR-DNN and Fuzzy Clustering for Subtype-Aware Drug Response Prediction in Triple Negative Breast Cancer

To address the therapeutic challenges in triple negative breast cancer (TNBC) driven by its molecular heterogeneity, a two-stage hybrid framework was developed. A multitask QSAR deep neural network (MTL-QSAR DNN) was constructed, featuring a subtype-aware masking mechanism to tailor predictions. The model was trained by integrating molecular descriptors generated by PaDEL-descriptor with genomic features from the Cell Model Passport database, which were first compressed via a variational autoencoder. In the second stage, the model's IC50 predictions across 26 biological pathways were translated into interpretable sensitivity profiles through Fuzzy C-Means (FCM) clustering, which was applied independently for each of the six TNBC subtypes. Strong predictive performance was demonstrated on the test data, with an R² of 0.81, and the resulting clusters were shown to be statistically significant. As a practical validation, ten leading candidates for drug repurposing were identified from a screen of 350 compounds.