Internal validation and explainability of machine learning models for abnormal liver biochemistry

INTRODUCTION: To develop, internally validate, and interpret machine learning models for classifying abnormal liver bio-chemistry using routine biochemical parameters.
METHODS: This retrospective study included 335 individuals classified according to ICD-coded diagnoses retrieved from the hospital information system, comprising 175 individuals in the non-disorder group and 160 individuals in the ab-normal liver biochemistry group. Aspartate aminotransferase, alanine aminotransferase, alkaline phosphatase, gam-ma-glutamyl transferase, total bilirubin, direct bilirubin, international normalized ratio, and albumin levels were used as predictors. Nine supervised learning models were evaluated using 10×5 repeated stratified cross-validation. The model with the numerically highest mean ROC-AUC was further examined using out-of-fold calibration analysis, exploratory decision curve analysis, and SHAP-based explanations.
RESULTS: Aspartate aminotransferase showed the highest mean individual ROC-AUC (0.832), followed by alanine ami-notransferase (0.796). XGBoost showed the numerically highest mean ROC-AUC (0.9640, 95% CI 0.9615–0.9664), with a mean accuracy of 0.9215 and F1-score of 0.9157. Mean sensitivity, specificity, positive predictive value, and negative predictive value were 0.9019, 0.9394, 0.9333, and 0.9161, respectively. Calibration analysis based on averaged repeated out-of-fold predictions yielded a Brier score of 0.056. SHAP analysis identified international normalized ratio, aspartate aminotransferase, and albumin as the most influential contributors to model predictions for the ICD-defined abnormal liver biochemistry outcome.
DISCUSSION AND CONCLUSION: XGBoost showed high internally validated classification performance for the ICD-defined abnormal liver biochemistry outcome. These findings should be interpreted as internally validated classification performance rather than definitive clinical diagnostic accuracy. Although the outcome was based on ICD-coded diagnoses rather than biochemical threshold rules, external validation using independently adjudicated clinical outcomes remains necessary.

Keywords: Artificial intelligence, biomarkers, cross-validation studies, liver function tests, machine learning, ROC curve