Benchmarking Machine Learning Models for Software Project Schedule Overrun Prediction

Abstract

Software projects frequently exceed their planned schedules because project performance is shaped by interacting technical, organizational, resource, risk, and stakeholder-related factors. This study benchmarks six regression algorithms—Linear Regression, Regression Tree, Random Forest, Support Vector Regression, Gradient Boosting Machine, and Deep Neural Network—to predict the percentage of schedule overrun in software projects and identify the model that provides the most favorable balance between predictive accuracy and interpretability. All six models were developed and evaluated using a publicly available dataset containing 4,517 software-project records. To ensure a consistent comparison, the models used identical predictors, default hyperparameter configurations, and a 10-fold cross-validation procedure. Predictive performance was assessed using root mean squared error, mean absolute error, correlation, and squared correlation. Gradient Boosting Machine achieved the lowest prediction errors, with an RMSE of 4.578 and an MAE of 3.503. Deep Neural Network produced nearly equivalent performance and recorded the highest squared correlation of 0.875. Random Forest ranked third, whereas Support Vector Regression produced the weakest performance under the adopted default settings. Interpretability analysis further identified Risk Factor, Duration Months, and Client Satisfaction Rating as the most consistently influential predictors across the interpretable models. The findings demonstrate the potential of ensemble and deep-learning methods to estimate the magnitude of schedule overrun using structured software-project data. Considering both prediction error and interpretability, Gradient Boosting Machine provided the strongest overall balance and represents a practical candidate for early-warning systems, risk-based project prioritization, schedule-recovery planning, and portfolio-level decision support. Future research should validate the models using independent organizational datasets, optimize their hyperparameters, and apply consistent explainability methods across all algorithms.