Rapid Pre-construction Forecasting of Schedule Overruns: A Machine Learning Regression Approach

1.1. Literature Review

Estimating the duration of a construction project is quite complex due to the existence of multiple interdependent activities, interdependent resources required, and the variability in crew productivity [18]. Various methods have been developed for this purpose, with differing levels of accuracy and complexity. The Critical Path Method (CPM), which is used by approximately 89% of U.S. contractors due to its simplicity, assumes deterministic values for activity durations, potentially compromising accuracy [19]. To account for potential delays, planners often add a “buffer” to the estimated duration, which can be estimated through comparison with similar past projects [13]. Limitations of CPM also include its failure to recognize shifting critical paths, which are typically managed by applying buffers that may not sufficiently reflect project-specific variables [11]. Some contractors calculate the buffer using a thorough risk analysis. This method results in much better accuracy, but it comes at the cost of a detailed analysis [19]. Only 33% of construction companies conduct such detailed analysis because of the skill and time needed to achieve results [12].

The Program Evaluation and Review Technique (PERT) enhances the CPM approach by incorporating three estimates for activity duration: optimistic, realistic, and pessimistic. This probabilistic framework allows for the calculation of the expected project duration and the likelihood of on-time completion. However, the complexity of gathering actual historical activity durations can deter project managers from using PERT, especially in large projects involving numerous activities [11, 19, 20]. The Probabilistic Network Evaluation Technique (PNET) improves upon PERT by accommodating the potential for any activity to impact the critical path, yet demands more extensive data input. Given the complexity of estimating durations for large construction projects with hundreds or thousands of activities, reliance on computer-based scheduling tools has become essential. Widely used software such as MS Project and Primavera P6 facilitates this process, with studies indicating that managers outside construction often prefer MS Project for its user-friendliness, while construction professionals favor Primavera P6 for its ability to manage complex projects [12, 14].

Schedule delays in building construction can result from a combination of factors: technical, contractual, managerial, and external factors. The most common factors reported in international studies include poor project management and supervision, inadequate planning and scheduling, design deficiencies and change orders, late delivery of materials, subcontractor performance issues, and owner-related problems such as unclear requirements or slow decisions [6, 21, 22]. Other factors like probabilistic and resource uncertainties, unforeseen site conditions, and inadequate risk allocation further increase schedule overruns [11, 13]. Institutional limitations and macroeconomic instability influence the causes of project delays. In developing countries, client and financing issues, administrative and permitting constraints, and supply-chain disruptions are frequently reported. Consequently, local context influences the relative importance of delay factors [6, 21].

Artificial Intelligence (AI) has found growing applications in construction [23]. Machine Learning (ML), a subset of AI, enables systems to learn and adapt over time. Despite the construction sector being one of the least digitalized industries, recent applications of AI and ML, such as estimating materials for construction [24], assessing labor safety risks [25], and predicting the performance of construction crews [26], demonstrate their growing potential. XGBoost, an ML algorithm [27], has been shown to be efficient and accurate in data analysis, especially in regression tasks [16]. Boosting methods improve predictive performance by combining successive decision trees to reduce model errors. XGBoost accommodates missing values and includes regularization mechanisms that help control overfitting. These capabilities support its application to heterogeneous construction data [28]. It uses K-fold cross-validation to enhance the accuracy of ML models by partitioning the dataset into K subsets, allowing for effective testing and training processes that utilize more data [16].

Many AI-based approaches to project scheduling classify projects by delay risk rather than estimate the magnitude of schedule overruns. Existing machine learning approaches differ in their methods and input requirements. For instance, fuzzy logic was used to adjust durations [18], and neural networks were used to successfully sort delay risks [29]. However, their data and input requirements may limit their use before construction begins. This study develops a compact model that uses readily obtainable pre-project inputs to provide construction teams with an estimate of delay magnitude.

1.2. Research Methodology

Extant reviews and meta-analyses were synthesized to compile a candidate list of delay factors commonly reported in the literature [6, 30]. The syntheses highlight recurring causes of delays such as client/owner issues, permitting/legal obstacles, design deficiencies, material and subcontractor delays, and project management shortcomings, which informed our selection of pre-project predictors [29, 31, 32].

Previous studies show that project context varies substantially, particularly across developing countries. Accordingly, the focus was on risk factors consistently reported across regions and project types [33]. Keeping the predictor set small and compact was a deliberate choice. Recent literature supports practical, data-driven tools that use information available before project commencement [34, 35]. Eleven primary delay factors were selected and expressed as measurable variables linked to specific survey questions (see Table 1). The confidential questionnaire was then distributed to several construction firms in Lebanon. Companies could submit data for one or more projects, resulting in a dataset of 60 completed building projects in Lebanon. The study was restricted to building projects because schedule delays depend on project type. Application to infrastructure projects or other regions would require additional data and model retraining. For variables representing supplier/subcontractor reliability, stakeholder performance, and external risks, the collected values were intended to characterize information available at project commencement, including historical performance/reliability and anticipated risks, rather than adverse events that subsequently occurred during project execution. For example, supplier and subcontractor variables represented their historical frequency of delayed performance, while municipality-related risk reflected the likelihood of problems expected at project commencement.

Extreme Gradient Boosting (XGBoost) was chosen for regression modeling due to its strong empirical performance, ability to combine many shallow decision trees into a robust ensemble, and effective handling of heterogeneous feature types and missing values.

XGBoost generates predictions by summing the contributions (leaf values) from each tree in the ensemble for a given observation. Each successive tree is trained to correct the residual errors of the preceding ensemble, producing a boosted model comprised of many trees. For prediction, a new observation is propagated through each tree, and the estimated delay is obtained as the sum of the contributions from the terminal leaves reached across the ensemble.

The dataset was partitioned into training (75%) and testing (25%) sets. Model selection and hyperparameter tuning were performed using 10-fold cross-validation on the training set. Final model performance was evaluated on the held-out testing set. To reduce sampling variability, the three-stage process (split → cross-validation training → test evaluation) was repeated 1000 times; reported metrics are the means across repetitions. These repetitions were used to assess the sensitivity and stability of model performance across different train–test partitions of the 60 available projects; they do not represent additional independent observations or external validation.

The data were separated into a model-development set and a final held-out test set before model training. Hyperparameter tuning was conducted exclusively within the model-development set using [cross-validation/ resampling/internal validation split]. The final test set was not used for hyperparameter selection, model comparison, feature selection, or any other modeling decision. After the hyperparameters and final model specification had been fixed, the model was evaluated once on the held-out test set using MAE and R².

Categorical and ordinal survey responses were numerically encoded for modeling. Examples: design quality (0 = poor, 1 = average, 2 = good), consultant performance (0 = poor, 1 = good), project complexity (0–2 scale), and legal/municipality issues (binary 0/1). Continuous variables (e.g., estimated duration, percentages) were retained in their native units.

1.3. Hyperparameter Tuning

Hyperparameter tuning was undertaken to maximize model generalization while avoiding overfitting [36]. Hyperparameter optimization was conducted in two stages. First, a coarse grid evaluated combinations of the principal XGBoost hyperparameters, including tree depth, learning rate, number of estimators, subsampling, column subsampling, and regularization parameters. This stage allowed the joint performance of representative hyperparameter combinations to be assessed. Second, one-dimensional sweeps were conducted around the best-performing region to refine selected parameters. This approach does not exhaustively characterize all higher-order interactions among hyperparameters, but it provides a computationally practical refinement strategy while retaining joint evaluation of the main parameter combinations in the initial stage. Model performance during tuning was assessed by 10-fold cross-validation on the training set and by mean MAE and R² across repeated randomized train/test partitions. This approach balanced exploration of the search space with computational tractability, reducing the risk of selecting values that overfit a single split.

Optimization robustness was assessed by repeating the complete train/test partitioning, cross-validation, and test evaluation procedure 1000 times with different randomized partitions. For each repetition, the tuning procedure was conducted using only the training portion, and performance was recorded on the corresponding held-out test portion. The resulting distributions of MAE and R² were summarized using their means, standard deviations, and 95% confidence intervals.

1.4. Dataset Summary and Descriptive Statistics

The dataset comprised 60 building projects. Key descriptive statistics and variable ranges are summarized in Table 3 (e.g., estimated duration, actual duration, delay days and percentage, supplier/subcontractor delay percentages, project manager experience, percentage subcontracted, days saved by crashing). Encoded categorical distributions are shown in Fig. (1) (design quality, consultant performance, project complexity, legal issues, and client performance).

The model was implemented in Python. The deployed tool prompts the user for 11 inputs corresponding to the selected predictors, as shown in Fig. (2). Outputs comprise: (1) a point estimate of project delay (percentage), (2) a 90% prediction interval computed from the empirical error distribution, and (3) the three features contributing most to the predicted delay for the given input. The 90% empirical prediction interval was constructed from the empirical distribution of prediction errors, defined as . The 5th and 95th percentiles of this error distribution were obtained and added to the point prediction, such that the interval for a prediction was . The resulting interval therefore represents an empirical estimate of predictive uncertainty rather than a parametric confidence interval.

1.5. Case Illustration

To demonstrate the model’s mechanics, a representative test observation was processed by the trained XGBoost ensemble composed of 70 trees. Each tree returned a leaf value for the observation; the predicted delay percentage is the sum of the leaf contributions across all trees. In the illustrative case, the ensemble predicted a 25% delay, compared with the observed 19%, demonstrating how the ensemble aggregates many small contributions to produce the final estimate.

A representative test case is presented to illustrate XGBoost prediction mechanics. The model was trained on the training set, and a single observation from the held-out test set was passed through the trained ensemble. The ensemble comprised 70 trees; Fig. (3) displays four example trees. Rather than relying on a single decision tree, the XGBoost predictor aggregates the contributions of all trees: each tree returns a leaf value for the observation, and the final predicted delay equals the sum of these leaf values. In this implementation, each tree considered three of the eleven available features, and more influential features were selected more frequently during tree construction. This ensemble approach produces more stable and accurate point estimates than a single decision tree.

A project from the testing dataset is chosen at random, and its parameters are shown in Table 4.

The test observation was propagated through all 70 trees to obtain the predicted delay. For the first four trees (Fig. 3), the leaf outputs were 0.6%, 0.6%, 0.1%, and 0.3%, respectively. The ensemble prediction is the sum of the leaf values returned by each tree. For this case, summing the contributions from trees 1–70 produced a predicted delay of 25% of the planned duration, as shown in Eq. (1). The observed delay for this project was 19%, so the residual for this example is 6 percentage points.

where ∇ is the predicted project delay and τ_i denotes the leaf contribution (in percent) from tree i

2.1. Interpretation of Predictor Effects

One predictor variable was removed at a time from the model input to evaluate its effect on the accuracy of the model by observing the change in mean absolute error on the test set. By examining Fig. (5), one can see how much the mean absolute error increased when each predictor factor was removed from the model input. The client (owner) performance and legal issues are identified as primary drivers of delay in the sample. Design quality and the Project Manager’s (PM) experience ranked third and fourth, respectively. These findings indicate that pre-project governance, client clarity and commitment, and design robustness are essential for reducing schedule risk. Supplier and subcontractor delay metrics and time savings from crashing activities proved to be less influential predictors for schedule delays. Because XGBoost captures nonlinear relationships and interactions among predictors, the reported importance values represent their relative contribution to predictive performance rather than fixed marginal effects or regression coefficients. Because the leave-one-predictor-out analysis evaluates the change in predictive error when a variable is omitted, the resulting ranking should be interpreted as predictive importance within the present dataset rather than as evidence of causal effects.

The leave-one-predictor-out analysis is a descriptive assessment of predictive importance; no inferential significance tests were used to assess model performance or predictor importance.

2.2. Practical implications

By requiring only eleven readily obtainable pre-project inputs, the model provides a practical decision support tool for project managers to estimate expected delay magnitude and its uncertainty before construction initiation. Outputs can inform mitigation strategies (e.g., contractual safeguards, design improvements, staffing or procurement interventions), scheduling adjustments, or negotiation of completion dates. While the model requires eleven user inputs, users may omit an input and still obtain an estimate, but predictive accuracy declines as more values are left unspecified. The highest-ranked predictors can be used as indicators of areas that may warrant additional managerial attention, risk review, or mitigation planning; however, their predictive importance should not be interpreted as evidence that modifying an individual factor will necessarily produce a causal reduction in project delay.

The following workflow describes how project teams can apply the pre-construction XGBoost delay model in practice to produce an adjusted completion forecast and prioritize mitigation actions. It assumes the user has a baseline schedule (e.g., CPM/Primavera) and can provide the simple pre-project inputs required by the tool.

1. Prepare baseline schedule: Compute the project’s estimated duration using standard scheduling tools (e.g., CPM/Primavera). Use this duration as the model’s “estimated duration” input.
2. Collect pre-project inputs: Gather the required pre-start predictors for the project: estimated duration (days), percentage of late supplier deliveries, percentage of subcontractor tasks delayed, project-manager years of experience, percentage of project subcontracted, days expected to be saved by crashing, design quality (0 poor / 1 average / 2 good), consultant performance (0 poor / 1 good), project complexity (0–2), presence of legal/ municipality issues (0/1), and client/owner performance (0–2). Use the same ordinal encodings as specified in Table 1.
3. Encode and validate inputs: Convert categorical/ordinal answers to numeric codes; verify numeric ranges and plausibility (e.g., percentage fields between 0–100). Correct or query obvious data entry errors before running the model.
4. Run the model: Input the eleven values into the tool. The model returns: (a) a point estimate of expected delay (percentage of estimated duration), (b) a 90% empirical prediction interval, and (c) the top three contributing factors ranked by their impact on the forecast.
5. Interpret and integrate outputs: Convert the predicted delay percentage to days and add to the baseline CPM estimate to obtain an adjusted expected completion date. Use the 90% interval to express forecast uncertainty to stakeholders. Use the ranked factors to prioritize mitigation actions (e.g., strengthen client payment/ communication protocols, accelerate permit resolution, improve design completeness, and assign a more experienced PM).
6. Decision and mitigation actions: Based on the adjusted schedule and top risk drivers, implement targeted mitigations (contract clauses, early procurement, design reviews, PM reassignment, contingency budgeting) or renegotiate milestones/contract terms if residual risk remains unacceptable.

In the case of large predicted delays or unusually wide prediction intervals, these often indicate either (a) true high risk (e.g., anticipated legal disputes, major financing uncertainty) or (b) an input configuration outside the model’s training distribution. In these cases, the user/project manager should: (1) treat the forecast as an alert requiring qualitative review; (2) inspect and verify inputs for errors; (3) perform a root-cause assessment focused on the top reported factors; and (4) consider bespoke risk analysis (detailed risk workshop, expert review, scenario planning) rather than relying solely on the model output.

2.3. Comparison with Prior Work

To show how this study advances the literature, the results are compared with those of previous studies that developed models to predict scheduling delays using large feature sets that include between 30 and 60 factors. Table 5 shows that this study advances the literature by (1) producing a regression model that estimates delay magnitude rather than class membership, and (2) achieving substantial explanatory power with a compact feature set (eleven predictors), thereby enhancing operational usability. The reported performance values are included for contextual completeness only. Because previous studies predominantly employ classification outcomes, whereas the present study employs regression, their classification accuracies are not directly comparable with the MAE and R² reported herein and should not be interpreted as evidence of relative predictive superiority.

The analysis of key input factors aligns with findings from previous studies: owner/client performance and legal/permit issues being the two strongest predictors of schedule delays, followed by design quality and project manager’s experience. Project complexity, consultant performance, and the share subcontracted had a moderate effect on the accuracy of the model, whereas supplier delays, days saved by crashing, and the proportion of subcontractor tasks experiencing delays had minimal effect.

It is worth noting that although many previous studies identified project manager’s experience as the single most influential factor, this factor ranked fourth in this model. This discrepancy is likely due to the use of years of experience of the project manager as a coarse operationalization to reflect the PM’s true competence. A more granular representation of managerial competence, such as qualifications, demonstrated past performance, or firm-level process metrics, would likely capture this factor’s influence more accurately and could change the relative ranking of the predictors.