Development of a LASSO machine learning algorithm-based model for postoperative delirium prediction in hepatectomy patients

Abstract

Objective

The objective of this study was to develop and validate a clinically applicable nomogram for predicting the risk of delirium following hepatectomy.

Methods

We applied the LASSO regression model to identify the independent risk factors associated with POD. Subsequently, we utilized R software to develop and validate a nomogram model capable of accurately predicting the incidence of POD.

Results

The final variables selected by the LASSO method were: Ramelteon, Age, Sex, Alcohol, Viral status, Cardiovascular disease, ASA class, Total bilirubin, Prothrombin time, Laparoscopic approach, and Blood transfusion. The performance of the nomogram was measured using ROC curve analysis, with an AUC of 0.854 (95% CI: 0.794–0.914) for the model. At the optimal cutoff value, the model demonstrated a sensitivity of 91.9% and a specificity of 68.8%. Model validation was performed using internal bootstrap validation to further verify the regression analysis. The ROC curve was generated by repeating the bootstrapping process 500 times, resulting in an AUC of 0.848 (95% CI: 0.786–0.904) for the model. The DCA curve representing the net benefit demonstrated the strong clinical validity of the model in predicting postoperative delirium.

Conclusion

Our results demonstrated that LASSO-based regression effectively constructed a nomogram model for predicting post-hepatectomy delirium.

Introduction

Hepatectomy is a prevalent procedure for treating hepatocellular carcinoma; however, post-hepatectomy delirium is a frequent postoperative complication with an incidence rate of approximately 17% [1]. Patients with post-hepatectomy delirium may experience cognitive decline, inattention, mood fluctuations, and even psychiatric abnormalities such as hallucinations and delusions in severe cases, making recovery and treatment extremely challenging. Furthermore, postoperative delirium typically manifests within a brief period following surgery, leading not only to extended hospital stay but also to an increased burden of care and elevated medical expenses [2]. Additionally, patients who suffer from severe postoperative delirium are at an elevated risk of death following surgery [3]. Some studies have demonstrated that early intervention in patients at a high risk of delirium can reduce the likelihood of its occurrence by approximately 50% [4]. Therefore, predicting the probability of post-hepatectomy delirium is crucial for timely intervention and treatment.

As a visual representation of the results of an analysis, a nomogram prediction model is used to predict the likelihood of a clinical outcome or an adverse event. This model boasts the advantages of being visually intuitive and user-friendly and has been widely employed in the evaluation of clinical diseases and prognoses. Nomograms have been demonstrated to effectively predict postoperative pulmonary complications in patients with diffuse peritonitis undergoing emergency gastrointestinal surgery [5]. Besides, nomograms have proven to be effective in predicting postoperative delirium in patients undergoing thoracic intra-aortic repair [6]. Furthermore, nomograms that combine factors such as age, type of surgery, electrolyte imbalance, and anemia can be used to predict postoperative delirium in patients undergoing hip arthroplasty [7]. However, there are relatively few nomogram models for predicting post-hepatectomy delirium.

Hence, the objective of this study was to develop and validate a clinically applicable nomogram for predicting the risk of delirium following hepatectomy in order to improve early prediction and intervention.

Methodology

Study population and clinical variables

A total of 306 patients undergoing hepatectomy were enrolled in this study.

A variety of variables were taken into account for this study, including the patient’s medical history, comorbidities, the American Society of Anesthesiologists (ASA) score, blood loss, transfusion utilization, and the duration of surgery. Those who had undergone biliary reconstruction or suffered from intestinal obstruction were excluded from the study. Postoperative delirium was evaluated using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) of the American Psychiatric Association. The DSM-5 identifies delirium by the following five factors: (A) a deficit in attention and awareness; (B) a brief onset of impairment; (C) additional impairment in cognition; (D) impairment in criteria A and C that cannot be better attributed to preexisting, established, or evolving neurocognitive impairment and does not occur in situations where the level of arousal is severely reduced, such as coma; and (E) evidence from history, physical examination, or laboratory findings that the disorder is a direct physiological consequence of another medical condition, substance intoxication or withdrawal.

Using LASSO regression method to screen the related variables of the model

LASSO (Least Absolute Shrinkage and Selection Operator) regression analysis is a statistical method used for variable selection, which reduces the model complexity by adding an absolute value penalty term on the basis of least squares regression and can select important variables. In the R software, LASSO regression analysis can be implemented through various packages, among which the most commonly used is the glmnet package. Here are the basic steps for LASSO regression analysis using the glmnet package: (1) Install and load the glmnet package; (2) Prepare available data; (3) Fit a LASSO regression model by setting the response variable as y and the explanatory variable matrix as x, and selecting the lambda parameter, which determines the strength of the penalty; (4) Extract important variables: lambda_min returns the optimal lambda value in cross-validation, etc.; (5) Get the coefficients of LASSO regression.

Construction and evaluation of nomogram models

Based on the optimal variables selected by the R software, the nomogram model was developed, and its performance was assessed utilizing ROC curves and AUC measurements. Using nomograms, a logistic regression linear model can be transformed into scores ranging from 0 to 100. The corresponding prediction probabilities were calculated by summing the scores for each variable. Furthermore, the model underwent internal validation running 500 resamples to confirm its accuracy, and its clinical validity was verified through decision curve analysis (DCA).

In this study, a nomogram prediction model for post-hepatectomy delirium was developed based on the screened independent risk factors. The screened predictors were consolidated to construct a nomogram prediction model for post-hepatectomy delirium. Users could sequentially select the endpoints of the corresponding line segments based on the conditions of each index and draw a vertical line upwards to the scoring axis to obtain an individual item score. The total score was then calculated by summing all individual score values. Upon locating the corresponding score point on the total score axis, a vertical line was drawn downwards to the bleeding risk axis to determine the incidence of post-hepatectomy delirium.

DCA (Decision Curve Analysis) determines the clinical utility of a nomogram model by calculating the net benefit under the probability threshold of postoperative delirium risk for each patient. The horizontal coordinate of DCA represents the probability threshold of high risk, while the vertical coordinate represents the net benefit (NB). The model assesses the rate of net benefit and the range of effective predictive probabilities by subtracting the false positive population due to model misclassification. When all patients have no postoperative delirium or all patients have postoperative delirium, the nomogram model has no clinical utility.

Statistical analysis

We used R version 3.1.3 (http://www.R-project) and Empower Stats (http://www.empowerstats.com/cn/download.html) for our analysis.Categorical data were presented as frequencies and percentages (%), and a χ2 test was employed to compare between groups. Normally distributed measurement data were expressed as the mean ‾± standard deviation (‾± s). The R package and the rms package were used to construct the nomogram prediction model. The caret package and the Bootstrap method were applied for internal validation. The consistency index (C-index) was calculated using the rms package. ROC curves were generated using the ROCR and rms packages. A difference was deemed statistically significant if P < 0.05.

Results

Delirium was observed in 34 patients following hepatectomy for hepatocellular carcinoma, while 272 patients did not experience delirium. There was a significant difference in the age between the two groups, but no difference in the duration of the operation (Table 1).

Table 1 Clinical characteristics of patients with postoperative delirium

To further mitigate confounding factors in the data, LASSO regression analysis was performed on preoperative indicators and intraoperative conditions variables.

Lambda selected by LASSO: Select lambda = lambda.min: 0.0128 (−4.3617).

The final variables selected by the LASSO method were: Ramelteon, Age, Sex, Alcohol, Viral status, Cardiovascular disease, ASA class, Total bilirubin, Prothrombin time, Laparoscopic approach, and Blood transfusion (Fig. 1).

Fig. 1

LASSO analysis: (A)-Demographic and clinical feature selection using the LASSO binary logistic regression model; (B)- Optimal parameter (lambda) selection in the LASSO model

Predictive model

logit(Postoperative delirium) = −5.20273 + 1.44061*Age − 2.30516*Sex + 0.80120*Alcohol + 1.99157*(Viral status = HCV) + 1.30474*(VIRAL.STATUS = None) + 0.73494*Cardiovascular disease + 0.27355*(ASA class = 2) + 1.23804*(ASA class = 3) −1.28758*Ramelteon + 1.16813*T.BIL + 1.31642*Blood transfusion − 0.01057*Prothrombin time − 0.31936*Laparoscopic approach.

Establishment of the nomogram model

The nomogram model was developed based on the variables identified through LASSO regression analysis (Fig. 2). Scoring can be performed for each patient according to the established risk factors. The higher the total score, the greater the likelihood of POD. This score enables us to make a preliminary prediction of the possibility of postoperative delirium. Clinicians can use these readily available indicators to assess the risk of postoperative delirium in a visual, personalized, and quantitative manner. A higher C-index indicates better discrimination by the model, suggesting greater accuracy in the nomogram predictions. The performance of the nomogram was measured using ROC curve analysis, with an AUC of 0.854 (95% CI: 0.794–0.914) for the model. At the optimal cutoff value, the model demonstrated a sensitivity of 91.9% and a specificity of 68.8% (Table 2 and Fig. 3A).

Fig. 2

The nomogram for predicting POD. (The values of each variable for an individual patient are plotted along each variable axis, and a line is drawn upwards to find the points received for each variable value. Then, the sum of these numbers is located on the total points axis, and a line is drawn downwards to the axis of risk to determine the likelihood of postoperative delirium.)

Table 2 ROC curve and best threshold analysis

Fig. 3

The receiver operating characteristic (ROC) curve of the model for predicting postoperative delirium. A The ROC curve of the full data prediction model. B The ROC curve of the full data prediction model after internal validation

Model validation was performed using internal bootstrap validation to further verify the stepwise regression analysis. The ROC curve was generated by repeating the bootstrapping process 500 times, resulting in an AUC of 0.848 (95% CI: 0.786–0.904) for the model, with statistical power similar to that of the initial model (Fig. 3B). The better the discrimination, the closer the C-index would approach 1. Therefore, the nomogram prediction model displayed exceptional discrimination.

Similarly, we used only preoperative variables to construct a predictive model for postoperative delirium, with the following results. The preoperative data prediction model was assessed for its diagnostic performance, resulting in an ROC AUC of 0.8601 with a 95% confidence interval ranging from 0.8015 to 0.9187. The model identified an optimal threshold of 0.1109, achieving a specificity of 0.7390 and a sensitivity of 0.8824. The accuracy of the prediction model was determined to be 0.7549, with a positive likelihood ratio of 3.3803 and a negative likelihood ratio of 0.1592. The diagnostic odds ratio for the preoperative data prediction model was calculated to be 21.2324, signifying its clinical predictive value (Table 2).

The DCA curve representing the net benefit demonstrated the strong clinical validity of the model in predicting postoperative delirium. The decision curve analysis (DCA) curve provides favorable net benefits over a wide range, with threshold probabilities for the model ranging from 5 to 80%. In conclusion, our model effectively predicted post-hepatectomy delirium outcomes by combining various assessment parameters (Fig. 4).

Fig. 4

The decision curve analysis (DCA) of the nomogram for predicting postoperative delirium risk

Discussions

Post-hepatectomy delirium is a pathological state characterized by symptoms such as impaired consciousness, cognitive dysfunction, psychiatric abnormalities, and behavioral disturbances following hepatectomy surgery. Delirium typically manifests within 24–72 h after surgery and ranges in severity from mild cases of inattention and slowed thinking to severe cases featuring hallucinations, delusions, agitation, depression, and other abnormal psychiatric symptoms, including loss of consciousness. Early diagnosis and treatment of delirium can reduce length of hospital stay, in-hospital morbidity, and healthcare costs [8]. The development of predictive models for post-hepatectomy delirium can assist physicians in identifying high-risk factors in advance and enhancing monitoring and intervention, thereby reducing the incidence and severity of post-hepatectomy delirium and improving the quality of medical care and services. Our results demonstrated that LASSO-based regression effectively constructed a nomogram model for predicting post-hepatectomy delirium. The nomogram model comprised the following variables: Ramelteon, Age, Sex, Alcohol, Viral status, Cardiovascular disease, ASA class, Total bilirubin, Prothrombin time, Laparoscopic approach, and Blood transfusion. The results indicated that the nomograms exhibited excellent discrimination and calibration effects and possessed strong clinical utility.

Some studies have demonstrated that ramelteon combined with or without sunvozertinib is the only significant preventive factor for delirium following pharyngotomy combined with esophagectomy [9]. A meta-analysis, which encompassed a total of 15 studies, revealed that age > 70 years and psychiatric medications were risk factors for delirium after hip surgery in elderly patients [10]. Other studies employing multivariate analysis identified independent predictors of postoperative delirium as 60–89 years of age, gender, and non-elective surgery [11]. Age and preoperative alcohol use have also been shown to be risk factors for postoperative delirium [12]. Additionally, studies utilizing the NIS database derived risk factors for postoperative delirium including advanced age, neurological disorders, alcohol and drug abuse, depression, psychosis, fluid and electrolyte disturbances, diabetes, and weight loss [13]. An ASA class of 3 and patients aged ≥ 74 years are at a risk of greater than 50% for developing delirium [14]. Furthermore, three variables were identified as independent predictors of postoperative delirium in multiple regression analysis: history of alcohol consumption, history of hepatic encephalopathy, and MELD score [15]. There may be an independent association between intraoperative autologous and allogeneic blood transfusion and delirium after complex spinal fusion [16]. In addition, a systematic review of 15 articles included in the analysis found that delirium was prevalent in between 5% and 39% of the studies, with many factors associated with an increased risk of delirium including age, cognitive impairment, comorbidities, smoking, alcohol use, visual acuity, ASA scores, and blood loss [17]. The findings of our study were in agreement with these conclusions.

The present study does indeed have several limitations, which can be elaborated as follows: Firstly, (1) the research participation model was constructed based on a dataset from a single medical center, which may affect the external validity of the model. It is imperative to further validate the dataset from other medical centers to assess the generalizability and robustness of the model. Secondly, (2) during the model construction process, the inclusion of proteomics data from blood samples was overlooked, which could potentially result in an incomplete understanding of the underlying pathophysiology of postoperative delirium. Future research should prioritize the analysis of blood proteomics to provide deeper insights into the biological basis of delirium. Moreover, incorporating more accurate liver function indicators, such as Indocyanine Green (ICG) and Aspartate Transaminase (ATI), could enhance the predictive accuracy and clinical utility of the model. Thirdly, (3) this study primarily focused on the incidence of short-term postoperative delirium, without assessing the mid-term postoperative cognitive function of patients. The evaluation of mid-term cognitive outcomes is crucial for understanding the long-term impact of delirium, and thus, future prospective follow-up studies should be conducted to comprehensively evaluate long-term prognosis. Furthermore, (4) the current body of research on the correlation between liver fibrosis and activity and postoperative delirium is limited. The lack of in-depth analysis of these pathological parameters within the postoperative pathological background may affect the applicability of the model. Future studies should explore the relationship between these factors and postoperative delirium. Lastly, (5) the study population was restricted to patients undergoing hepatectomy, without a specific examination of postoperative delirium in different subgroups. This limits the general applicability of the model. Therefore, future research should be expanded to include patients undergoing various types of surgery, particularly those with different underlying diseases or physiological conditions, to validate the model’s applicability and precision across diverse patient populations.

The nomogram chart provides a visual representation of risk assessment for clinicians and researchers. It enables them to estimate the probability of specific clinical outcomes based on a combination of baseline characteristics and potential risk factors. By employing predictive models, clinicians can quantify and integrate these risks into their decision-making process for individualized treatment or when considering intervention strategies to avert adverse outcomes [18, 19].

While we acknowledge that some of the variables included in the nomogram may be considered baseline characteristics rather than direct clinical practices (with exceptions such as laparoscopic examination and transfusion), it is important to remember that baseline characteristics can serve as important predictors of clinical outcomes. By incorporating these variables into the nomogram, we aim to provide a comprehensive tool that considers multiple factors influencing patient outcomes.

Furthermore, for patients identified as high risk by the model, we can adopt more proactive preventive interventions, such as:

• Preoperative education: Providing comprehensive education to patients prior to surgery to enhance their awareness and self-management capabilities regarding potential postoperative complications.

• Personalized medication and analgesia: Adjusting medication strategies based on the specific conditions of the patient avoid the use of drugs that may increase the risk of delirium.

• Encouraging family involvement: Engaging family members in the patient’s rehabilitation process to provide emotional support and assist in monitoring the cognitive status of the patient, among other measures.

In conclusion, this study constructed a nomogram prediction model of postoperative delirium in patients undergoing hepatectomy for hepatocellular carcinoma by LASSO regression based on clinical variables readily available to patients undergoing hepatectomy. As a result, it exhibited better predictive effects and could serve as a reference for appropriate clinical decision-making options, thus improving our clinical effectiveness and quality of care.