An Empirical Ablation Study of Activation Functions in U-Net for Colorectal Polyp Segmentation

Early detection of colorectal polyps is crucial to prevent progression into cancer. However, accurate detection remains challenging, especially for small, flat, or hidden polyps, and in images with poor visual quality or uneven illumination. U-Net-based segmentation has been widely adopted for its ability to preserve spatial details, yet its effectiveness is highly influenced by internal configurations-particularly the activation function, which governs data flow and non-linear representation. An inappropriate choice may lead to the loss of critical signals and suboptimal segmentation outcomes. This study presents an empirical ablation study of seven activation functions-ReLU, LeakyReLU, ELU, SELU, GELU, Swish, and Softplus-applied within the U-Net architecture. The evaluation was conducted using four key quantitative metrics (Accuracy, Dice, IoU, and Loss), along with visual analysis to assess each function's response to variations in polyp morphology and image quality. The findings show that activation functions like Leaky ReLU, ELU, SELU, and Softplus perform poorly and are unstable for colorectal polyp segmentation under visual variations. ReLU exhibited stable performance and was effective in identifying polyps with sharp contours and high contrast but was less adaptive to subtle regions. Swish demonstrated consistent training behavior and adaptability to irregular shapes and uneven lighting, yet its overall performance was still behind the best-performing function. GELU achieved the highest performance, excelling in detecting small, smooth polyps and showing robustness against noise and visual artifacts. Among all, GELU emerges as the most promising activation function for achieving accurate, robust, and consistent colorectal polyp segmentation.