Evaluating the Robustness of Mean Decrease Accuracy and Mean Decrease Gini in Random Forest Classification under Multicollinearity

This paper evaluates the robustness of two widely used VIMs in Random Forest classification-MDA and MDG-under varying levels of multicollinearity and sample sizes. A controlled simulation framework was designed with twelve predictors grouped into high, moderate, and no correlation structures, across two sample sizes (n=100 and n=1000). In addition, a real-world dataset comprising 943 firms listed on the IDX in 2024 was analyzed, where financial distress status was defined using the ICR and ROA. The evaluation employed rank correlation and overlap-based metrics to measure the agreement between variable importance rankings and groundtruth or literature-supported benchmarks. The simulation results show that MDA more consistently recovers the true ranking of influential variables compared to MDG, although both suffer from distortions under strong multicollinearity. In the real financial distress dataset, both methods identified Earning to Debt (ETD) and Net Profit Margin (NPM) as the most important predictors, consistent with established literature. Overall, MDA demonstrates greater robustness than MDG in classification settings with correlated predictors, but further improvements are needed through advanced techniques to fully mitigate correlation-induced bias.