Comparison of an interpretable extreme gradient boosting model and an artificial neural network model for prediction of severe acute pancreatitis

Abstract

Introduction: Acute pancreatitis (AP) that progresses to persistent organ failure is referred to as severe acute pancreatitis (SAP). It is a condition associated with a relatively high mortality. A prediction model that would facilitate early recognition of patients at risk for SAP is crucial for improvement of patient prognosis.

Objectives: The aim of this study was to evaluate the accuracy of extreme gradient boosting (XGBoost) and artificial neural network (ANN) models for predicting SAP.

Patients and methods: A total of 648 patients with AP were enrolled. XGBoost and ANN models were developed and validated in the training (519 patients) and test sets (129 patients). The accuracy and predictive performance of the XGBoost and ANN models were evaluated using both the area under the receiver operating characteristic curves (AUCs) and the area under the precision‑recall curves (AUC‑PRs).

Results: A total of 15 variables were selected for model construction through a univariable analysis. The AUCs of the XGBoost and ANN models in 5‑fold cross‑validation of the training set were 0.92 (95% CI, 0.87–0.97) and 0.86 (95% CI, 0.78–0.92), respectively, whereas the AUCs for the test set were 0.93 (95% CI, 0.85–1) and 0.87 (95% CI, 0.79–0.96), respectively. The XGBoost model outperformed the ANN model in terms of both diagnostic accuracy and AUC‑PR. Individual predictions of the XGBoost model were explained using a local interpretable model‑agnostic explanation plot.

Conclusions: An interpretable XGBoost model showed better discriminatory efficiency for predicting SAP than the ANN model, and could be used in clinical practice to identify patients at risk for SAP.