Comparative Evaluation of Loss Function for 3D Brain Vessel Segmentation in MRA Using U-NET

Cerebrovascular segmentation using three-dimensional Time-of-Flight Magnetic Resonance Angiography (TOF-MRA) is crucial for diagnosing and assessing neurovascular diseases. However, accurate segmentation remains challenging because of severe class imbalance, complex vascular topology, and the presence of thin and peripheral vessels. This study presents a systematic empirical evaluation of loss function design for 3D brain vessel segmentation using a standard U-Net architecture. An initial comparison of three commonly used loss formulations Dice Loss, BCE + Dice, and Weighted BCE + Dice is conducted, followed by an extended evaluation including widely adopted imbalance aware and topology-related loss functions such as Tversky, Focal Tversky, Generalized Dice, Boundary Loss, and ClDice, all under identical experimental settings. To ensure a robust and reliable assessment, a fivefold stratified cross-validation strategy was employed. Segmentation performance was evaluated using overlap-based metrics (Dice/F1 Score and Intersection-over-Union), voxel-wise classification measures (precision and recall), and clinically relevant error metrics, including false-positive rate (FPR), false-negative rate (FNR), and receiver operating characteristic area under the curve (ROC-AUC). The experimental results demonstrate that the Weighted BCE + Dice loss consistently achieves higher average performance with lower variance across cross-validation folds, offering a favorable balance between overlap accuracy, sensitivity to small-caliber vessels, and clinical error control. Qualitative visualization further confirms improved vessel continuity and completeness, particularly in thin and peripheral branches. Rather than proposing a novel architecture or claiming state-of-the-art performance, this study provides a controlled and reproducible empirical baseline for loss function selection in highly imbalanced 3D cerebrovascular segmentation. The findings offer practical guidance for designing robust segmentation pipelines and support future research on advanced architectural and topology-aware methods for TOF-MRA vessel analysis.