- Research article
- Open Access
- Open Peer Review
Predicting the risk of death following coronary artery bypass graft made simple: a retrospective study using the American College of Surgeons National Surgical Quality Improvement Program database
Journal of Cardiothoracic Surgeryvolume 10, Article number: 62 (2015)
Risk models to predict 30-day mortality following isolated coronary artery bypass graft is an active area of research. Simple risk predictors are particularly important for cardiothoracic surgeons who are coming under increased scrutiny since these physicians typically care for higher risk patients and thus expect worse outcomes. The objective of this study was to develop a 30-day postoperative mortality risk model for patients undergoing CABG using the American College of Surgeons National Surgical Quality Improvement Program database.
Material and methods
Data was extracted and analyzed from the American College of Surgeons National Surgical Quality Improvement Program Participant Use Files (2005–2010). Patients that had ischemic heart disease (ICD9 410–414) undergoing one to four vessel CABG (CPT 33533–33536) were selected. To select for acquired heart disease, only patients age 40 and older were included. Multivariate logistic regression analysis was used to create a risk model. The C-statistic and the Hosmer-Lemeshow goodness-of-fit test were used to evaluate the model. Bootstrap-validated C-statistic was calculated.
A total of 2254 cases met selection criteria. Forty-nine patients (2.2%) died within 30 days. Six independent risk factors predictive of short-term mortality were identified including age, preoperative sodium, preoperative blood urea nitrogen, previous percutaneous coronary intervention, dyspnea at rest, and history of prior myocardial infarction. The C-statistic for this model was 0.773 while the bootstrap-validated C-statistic was 0.750. The Hosmer-Lemeshow test had a p-value of 0.675, suggesting the model does not overfit the data.
The American College of Surgeons National Surgical Quality Improvement Program risk model has good discrimination for 30-day mortality following coronary artery bypass graft surgery. The model employs six independent variables, making it easy to use in the clinical setting.
Developing risk models to predict 30-day postoperative mortality following isolated coronary artery bypass graft (CABG) has been an active area of research [1-3]. No such risk model has been developed using the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database. This multi-institutional database was created to provide clinical data for improving the quality of surgical outcomes [4,5]. ACS NSQIP now obtains data from more than 525 participating hospitals covering approximately 30% of the operative volume of the United States [6,7]. Demographic, preoperative comorbidity, operative data, and 30-day postoperative morbidity and mortality data is collected in a systematic and standardized manner for use in outcomes research . The objective of this study was to develop a 30-day postoperative mortality risk model for isolated CABG utilizing ACS NSQIP, simplified for clinical practice.
Exempt status was granted from our institution’s internal review board. Data was taken from the 2005–2010 ACS NSQIP Participant Use Files, including 240 demographic, preoperative, operative, and morbidity/mortality variables. Diseases and procedures are classified using International Classification of Diseases 9 Clinical Modification (ICD-9-CM) and Current Procedural Terminology (CPT) codes.
Patients who underwent CABG using one to four vessel arterial grafts (CPT codes 33533–33536) were included in the study. These codes also include coronary artery bypass procedures using arterial grafts only, or a combination of arterial-venous grafts . To focus on ischemic heart disease, only patients with a diagnosis of ischemic heart disease (ICD-9-CM codes 410–414) were included. To select for acquired heart disease, only patients 40 years and older were included. A total of 66 variables that covered demographic, comorbidity, preoperative laboratory values, previous medical/surgical interventions, and perioperative data were used in the analysis.
Univariate analysis was performed with p < 0.2 as the inclusion criteria. Multiple imputation was performed on continuous variables. Variables with more than 5% of values missing were not included in univariate analysis . The Wilcoxon rank sum test and Pearson’s X2 test were used for continuous and categorical variables respectively. Stepwise backwards selection was used on the outcome of univariate analysis and candidate variables were obtained (Table 1) for multivariate logistic regression analysis. Variables with p < 0.05 were candidates for inclusion in the final multivariate model, however those with wide 95% confidence interval were excluded. The C-statistic was used to determine the model’s discriminative ability . The bootstrap method was used to find the optimism-corrected C-statistic . The Hosmer-Lemeshow statistic was used to determine goodness of fit . A risk score was then created using the final multivariate statistical model. Statistical analysis was performed using R version 3.0 .
From 2005–2010, a total of 2254 cases fitting the inclusion criteria were found from a total of 4317 CABGs recorded during that time period. Most patients were male (74.8%, n = 1687), with a median age of 66.0 years. The incidence of mortality within 30-days of surgery was 2.2% (n = 49), involving mostly males (67.3%, n = 33), and a median age of 70.8 years.
The univariate analysis results are shown in Table 1. Although a cutoff of p < 0.2 was chosen, current literature supported the addition of sex, body mass index (BMI), and previous percutaneous coronary intervention (PCI) to multivariate analysis [2,15,16]. In multivariate analysis, previous PCI was found to be marginally significant with p = 0.0648, however it was kept because the C-statistic of the final model decreased from 0.773 to 0.762, suggesting its importance. Of note, history of ischemic rest pain/gangrene, renal failure, transfusions of four or more units of red blood cells, and do not resuscitate status were not included in the final model because of wide 95% confidence intervals. Duration of operation was significant but excluded, without detriment to the model’s discriminative ability, to ensure all variables could be obtained preoperatively. The final model contained only six variables (Table 2).
The C-statistic for the final model was 0.773, demonstrating good discriminative ability (Figure 1). The optimism-corrected C-statistic found using the bootstrap method was 0.750. The Hosmer-Lemeshow statistics also suggested that the model did not over fit the data (p = 0.675).
A risk score was created using the β coefficients of logistic regression model. The risk score ranged from 1 to 23 (Table 3). The lowest observed risk score from the data was two while the highest was 19 (Figure 2). The probability of 30-day mortality ranged from 0.14% for patients with a score of two, to 58.0% for those with a score of 19. The mean probability of mortality was 2.2%, the median was 1.3%, and the standard deviation was 3.0%. A score of eight or less corresponded to a predicted mortality less than the mean, which included 1860 patients (82.5%), while a score of nine approximately corresponded to the mean probability of mortality. Twenty-one patients (0.9%) had a predicted mortality greater than two standard deviations from the mean probability (8.2%), which corresponded to a risk score of 12 or greater.
We present a simple risk score to estimate 30-day mortality following isolated CABG with discrimination comparable to other major risk calculators in cardiac surgery [1-3]. To assess outcomes and to provide patients and families with realistic predictions of operative risk, the need for simple preoperative risk calculators remains ever present. With outcome data publically available, risk assessment tools are necessary for cardiothoracic surgeons, who are coming under increased scrutiny . Risk scores can “level the playing field,” to fairly assess outcomes for surgeons who care for higher risk patients and therefore expect worse outcomes.
The ACS NSQIP model shares many similarities with other risk models such as EuroSCORE , the New York Risk Score [3,18], and the Society of Thoracic Surgeons (STS) 2008 Cardiac Surgery Risk Model . Shared variables include age, previous MI, PVD, renal failure, hemodynamic state and ejection fraction (EF). An ad hoc committee concluded that seven core variables should be present in any database reporting risk-adjusted outcomes for CABG which include age, sex, previous heart operation, EF, percent stenosis of left main coronary artery, number of major coronaries with greater than 70% stenosis and level of acuity . In the STS model, 78% of the variance is explained by eight of the most important variables, which include age, surgical acuity, reoperative status, creatinine level, dialysis, shock, chronic lung disease, and EF . While not a specialty database tailored to specific procedures, the ACS NSQIP database includes all of these core variables except for EF and other cardiac variables. Our model includes several variables listed in the consensus statement and other CABG datasets (age, previous MI, pervious PCI) while also finding ischemic rest pain/gangrene (a proxy for peripheral vascular disease), renal failure, and preoperative transfusion (a proxy for preoperative anemia) significant, but ultimately excluded due to their high confidence intervals. Preoperative anemia, while not a core variable, is a known risk factor for mortality following CABG .
Estimating risk involves using discrete markers as surrogates for the extent of cardiac disease. Core variables attempt to encapsulate the extent of a patient’s cardiovascular disease, the impact of that disease on end organ function, and ultimately reserve. BUN addresses not only the degree of renal dysfunction, but also the quality and frequency of hemodialysis—important since the impact of uremia on cardiac function is well known . We argue that in recent times, previous PCI might be more valid than previous cardiac surgery, in qualifying the severity of cardiac disease. Finally, hyponatremia is increasingly recognized as a predictor of postoperative morbidity and mortality following both cardiac and general surgery . Crestanello et al.  found preoperative hyponatremia as a risk factor for increased long-term morbidity, prolonged hospitalization, and mortality in patients undergoing cardiac surgery.
The ACS NSQIP model includes variables not yet included in specialty databases, but are known risk factors for mortality following CABG, such as those covering liver disease . This model covers a large subset of core variables common to the major risk models, without sacrificing discriminative ability. We argue that using a non-specialty database, while controversial, may prove beneficial to a field where risks models all share similar variables and “room for improvement for discriminating adverse outcomes may be limited .”
There are several limitations to this study. First, this is a retrospective study with data obtained by dedicated staff in a systematic fashion. Second, sample size was another limitation as there were fewer patients available to create the risk model compared to other specialty databases. A likely reason as to why there were fewer CABG cases during the 6 year period over which the study is employed is that sites submitting data to an established cardiac database could request for exemption for submission of cardiac subspecialty data . The paucity of data is perhaps secondary to the influence of other specialty databases, such as the STS ACSD . Third, cardiac-related variables were not available for inclusion in the final model. As a more generalized database, ACS NSQIP does not record all variables recommended in the consensus statement on the development of risk calculators . Despite these limitations, the ACS NSQIP risk model shows a discriminative ability on par with other available risk models.
In conclusion, our risk model provides cardiothoracic surgeons with a simple to use tool for assessing risk of mortality following CABG using six preoperative, independent variables that can be assessed with history, physical exam and a serum chemistry panel. Finally our study demonstrates the versatility of the ACS NSQIP database.
Coronary Artery Bypass Graft
- ACS NSQIP:
American College of Surgeons National Surgical Quality Improvement Program
Internal Board Review
International Classification of Disease
Current Procedural Terminology
Receiver Operating Characteristic
Body Mass Index
Nashef S, Roques F. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg. 1999;16:0–4.
Shahian DM, O’Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB, et al. The Society of Thoracic Surgeons 2008 Cardiac Surgery Risk Models: part 1-Coronary Artery Bypass Grafting Surgery. Ann Thorac Surg. 2009;88(1 Suppl):S2–22. doi:10.1016/j.athoracsur.2009.05.053.
Hannan EL, Farrell LS, Wechsler A, Jordan D, Lahey SJ, Culliford AT, et al. The New York risk score for in-hospital and 30-day mortality for coronary artery bypass graft surgery. Ann Thorac Surg. 2013;95(1):46–52. doi:10.1016/j.athoracsur.2012.08.047.
Khuri SF, Daley J, Henderson W, Hur K, Gibbs JO, Barbour G, et al. Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J Am Coll Surg. 1997;185(4):315–27.
Khuri SF, Daley J, Henderson W, Hur K, Demakis J, Aust JB, et al. The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998;228(4):491–507.
Cohen ME, Ko CY, Bilimoria KY, Zhou L, Huffman K, Wang X, et al. Optimizing ACS NSQIP Modeling for Evaluation of Surgical Quality and Risk: Patient Risk Adjustment, Procedure Mix Adjustment, Shrinkage Adjustment, and Surgical Focus. J Am Coll Surg. 2013; doi:10.1016/j.jamcollsurg.2013.02.027.
Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY, et al. Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217(5):833–42. e3. doi:10.1016/j.jamcollsurg.2013.07.385.
Rowell KS, Turrentine FE, Hutter MM, Khuri SF, Henderson WG. Use of national surgical quality improvement program data as a catalyst for quality improvement. J Am Coll Surg. 2007;204(6):1293–300. doi:10.1016/j.jamcollsurg.2007.03.024.
American Medical Association, American College of Cardiology Foundation. CPT Reference Guide for Cardiovascular Coding 2013. 1st ed. (American Medical Association, ed.). American Medical Association; 2012:326.
Hamilton BH, Ko CY, Richards K, Hall BL. Missing data in the American College of Surgeons National Surgical Quality Improvement Program are not missing at random: implications and potential impact on quality assessments. J Am Coll Surg. 2010;210(2):125–39. e2. doi:10.1016/j.jamcollsurg.2009.10.021.
Merkow RP, Hall BL, Cohen ME, Dimick JB, Wang E, Chow WB, et al. Relevance of the c-statistic when evaluating risk-adjustment models in surgery. J Am Coll Surg. 2012;214(5):822–30. doi:10.1016/j.jamcollsurg.2011.12.041.
Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453–73.
Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York: Wiley-Interscience Publication; 2000. p. 392.
R Core Team. R: A Language and Environment for Statistical Computing. 2013.
Jones RH, Hannan EL, Hammermeister KE, Delong ER, O’Connor GT, Luepker RV, et al. Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. The Working Group Panel on the Cooperative CABG Database Project. J Am Coll Cardiol. 1996;28(6):1478–87.
Hannan EL, Racz M, Culliford AT, Lahey SJ, Wechsler A, Jordan D, et al. Risk score for predicting in-hospital/30-day mortality for patients undergoing valve and valve/coronary artery bypass graft surgery. Ann Thorac Surg. 2013;95(4):1282–90. doi:10.1016/j.athoracsur.2012.11.019.
Burack JH, Impellizzeri P, Homel P, Cunningham JN. Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. Ann Thorac Surg. 1999;68(4):1195–200. discussion 1201–2.
Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP, et al. Risk stratification of in-hospital mortality for coronary artery bypass graft surgery. J Am Coll Cardiol. 2006;47(3):661–8. doi:10.1016/j.jacc.2005.10.057.
Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, et al. Cardiac surgery risk models: a position article. Ann Thorac Surg. 2004;78(5):1868–77. doi:10.1016/j.athoracsur.2004.05.054.
Williams ML, He X, Rankin JS, Slaughter MS, Gammie JS. Preoperative hematocrit is a powerful predictor of adverse outcomes in coronary artery bypass graft surgery: a report from the Society of Thoracic Surgeons Adult Cardiac Surgery Database. Ann Thorac Surg. 2013;96(5):1628–34. discussion 1634, doi:10.1016/j.athoracsur.2013.06.030.
Hung J, Harris PJ, Uren RF, Tiller DJ, Kelly DT. Uremic cardiomyopathy–effect of hemodialysis on left ventricular function in end-stage renal failure. N Engl J Med. 1980;302(10):547–51. doi:10.1056/NEJM198003063021003.
Leung AA, McAlister FA, Rogers SO, Pazo V, Wright A, Bates DW. Preoperative hyponatremia and perioperative complications. Arch Intern Med. 2012;172(19):1474–81. doi:10.1001/archinternmed.2012.3992.
Crestanello JA, Phillips G, Firstenberg MS, Sai-Sudhakar C, Sirak J, Higgins R, et al. Preoperative hyponatremia predicts outcomes after cardiac surgery. J Surg Res. 2013;181(1):60–6. doi:10.1016/j.jss.2012.06.004.
Wang TKM, Li AY, Ramanathan T, Stewart RAH, Gamble G, White HD. Comparison of four risk scores for contemporary isolated coronary artery bypass grafting. Heart Lung Circ. 2014;23(5):469–74. doi:10.1016/j.hlc.2013.12.001.
ACS NSQIP. Participant Use Data File. Available at: http://site.acsnsqip.org/wp-content/uploads/2012/03/ACS-NSQIP-Participant-User-Data-File-User-Guide_06.pdf. Accessed July 5, 2014.
The authors declare that they have no competing interests.
Study concept and design: PJC, GS. Acquisition of data: PJC, ST, GS. Analysis and interpretation of data: PJC, ST, TIC, GS. Drafting of manuscript: PJC, TIC, JHB, GS. Critical revision of manuscript: PJC, TIC, JHB, AA, GS. All authors read and approved the final manuscript.