Skip to main content

Development of a predictive model for major adverse cardiac events in a coronary artery bypass and valve population



Quality improvement initiatives in cardiac surgery largely rely on risk prediction models. Most often, these models include isolated populations and describe isolated end-points. However, with the changing clinical profile of the cardiac surgical patients, mixed populations models are required to accurately represent the majority of the surgical population. Also, composite model end-points of morbidity and mortality, better reflect outcomes experienced by patients.


The model development cohort included 4,270 patients who underwent aortic or mitral valve replacement, or mitral valve repair with/without coronary artery bypass grafting, or isolated coronary artery bypass grafting. A composite end-point of infection, stroke, acute renal failure, or death was evaluated. Age, sex, surgical priority, and procedure were forced, a priori, into the model and then stepwise selection of candidate variables was utilized. Model performance was evaluated by concordance statistic, Hosmer-Lemeshow Goodness of Fit, and calibration plots. Bootstrap technique was employed to validate the model.


The model included 16 variables. Several variables were significant such as, emergent surgical priority (OR 4.3; 95% CI 2.9-7.4), CABG + Valve procedure (OR 2.3; 95% CI 1.8-3.0), and frailty (OR 1.7; 95% CI 1.2-2.5), among others. The concordance statistic for the major adverse cardiac events model in a mixed population was 0.764 (95% CL; 0.75-0.79) and had excellent calibration.


Development of predictive models with composite end-points and mixed procedure population can yield robust statistical and clinical validity. As they more accurately reflect current cardiac surgical profile, models such as this, are an essential tool in quality improvement efforts.

Peer Review reports


Quality improvement initiatives (QI), a cornerstone of cardiac surgery, have largely relied on predictive models to advance the quality of care cardiac surgical patients receive. For the last two decades, coronary artery bypass grafting (CABG) has dominated clinical practice in cardiac surgery, and therefore the majority of quality improvement initiatives have focused on surgical outcomes following isolated CABG surgery [14]. More recently, however, there has been an increase in valve and valve-CABG cases in cardiac surgery [5] and existing predictive models for isolated CABG may not accurately reflect current practice profiles. These models may achieve great statistical validity but lack clinical validity as the models are applicable to only a decreasing percentage of the clinical practice. In order to maintain continued success in quality improvement, it is important to delineate risk profiles for a group of mixed procedures, including CABG, valve, and valve + CABG, that characterize current clinical practice.

To accommodate the shift in the profile of cardiac surgical practice, isolated valve and valve plus CABG models have been developed [510]. Some debate exists regarding the validity of developing models with hetero-geneous procedures with some advocating for single procedure models [7, 9, 10]. Homogeneity within the population allows for simplicity of model development with improved reliability but limits sample size available for development and validation [5]. Although mixed procedural models may be confounded or biased by different pathophysiological and risk profiles, they improve sample size [5] and increase their relevance to current surgical practice.

Largely, mixed procedural models have focused on aortic/mitral or valve +/− CABG, with few CABG, valve and CABG +valve models present in the literature. The EuroSCORE model, one of the mostly widely used cardiac surgical prognostic models, has successfully achieved both statistical and clinical relevance as it applies to a mixed surgical practice including CABG, valve, and CABG + valve. However its development and validation cohort were derived from a population sample that is over fifteen years old [11] limiting its applicability to a contemporary surgical practice. Recently, the EuroSCORE II updated its risk model obviating some of the limitations of the original model [12]. However, the performance of this heterogeneous procedure model performed poorly in risk prediction for several of its component procedures types [13, 14]. In addition, both the EuroSCORE and the EuroSCORE II are only a mortality model.

Many models describe death as a lone end point, despite that fact that cardiac patients experience a variety of relevant morbidity. These alternate surgical outcomes are also important quality indicators for cardiac surgical care [15] and targets for QI initiatives. Subsequently, modeling composite end points such as, major adverse cardiac events (MACE), that included both important morbidity and mortality may provide more insight to the surgical outcomes experienced by patients as well as increase statistical power in low frequency end points.

Comprehensive models that include most major cardiac surgical procedures may allow for better understanding of patient risk profiles and facilitate quality improvement initiatives directed towards the majority of patients seen in current practice. The objective of this paper is to develop a CABG, valve, CABG + valve morbidity and mortality (MACE) model that can be applied to the majority of patients undergoing cardiac surgery.


Data source and study population

This study is a retrospective cohort design. The Maritime Heart Center Cardiac Surgery Registry is a detailed clinical database housed at the Queen Elizabeth II Health Science Center (QEII HSC) Halifax, Nova Scotia, an academic tertiary care centre performing 1200 cardiac cases per year. It includes pre-, intra-, and post-operative data prospectively collected on all cardiac surgical cases performed at the QEII HSC from 1995 to present. Trained abstractors collect data, and a database administrator maintains the registry. The database is audited annually.

The model development cohort included all patients undergoing isolated coronary artery bypass grafting (CABG), isolated aortic valve replacement, isolated mitral valve repair or replacement with or without concomitant CABG performed at the QEII HSC since 2004 to 2009. The development cohort was restricted to these years in order to maintain a modern population relevant to current clinical practice.

The primary outcome was a composite end point defined as Major Adverse Cardiac Events (MACE) that included in-hospital death, stroke (persisting at discharge or transient), acute renal failure (new post-operative renal failure or acute on chronic (>50% increase from baseline creatinine)), or infection (sepsis, pneumonia, or deep sternal wound infection).

Statistical methods

Variable selection

Candidate variables for model development included the following preoperative characteristics: age, sex, diabetes, frailty, chronic obstructive pulmonary disease, redo sternotomy, atrial fibrillation, hemoglobin, peripheral vascular disease, cerebro-vascular disease, creatinine, ejection fraction < 40%, New York Heart Association classification (NYHA I-IV) and body mass index (BMI). Surgery-related data such as urgency of surgery, and procedure type were also included. These candidate variables were chosen a priori and selected through rigorous review of the literature [19, 1618] (Table 1).

Table 1 Model variable definitions

Multi-colinearity of candidate variables was assessed with variance inflation factor (VIF) (Additional file 1: see Statistical Methods). The linear relationships of the natural variables and their transformations were assessed through locally weighted scatterplot smoothing (LOESS) regression [19] (see Additional file 1: Linearity and Transformations and Figure 2-3). The WHO classification of BMI was used [20].

Model evaluation

A multivariate logistic regression analysis was used to describe MACE.

Age, sex, procedure type and surgical priority were chosen a priori and forced into the model. Stepwise selection was implemented for the remainder of the candidate variables. The concordance statistic and −2 log likelihood were evaluated to assess the contribution of each variable to the model. If a variable did not contribute to an increase in the C or −2 log likelihood statistic it was not retained in the model.

Model discrimination was determined using the concordance statistic [21]. Model calibration was assessed by the Hosmer-Lemeshow goodness-of-fit statistic [22, 23] as well as calibration plots [24]. Deciles of observed and predicted probabilities of MACE were plotted for the calibration plots [5]. Bootstrap procedure was used to internally validate the model.

All statistical analysis was performed using SAS software version 9.2 (SAS, Cary, NC).

Approval for conducting this study was obtained from the Institutional Review Board of the Capital District Health Authority. The requirement to obtain informed consent was waived under Section 2.1c of the Tri-Council Policy Statement. All personal identifiers were stripped prior to data analysis to ensure patient anonymity and confidentiality.

The authors had full access to the data and take full responsibility for its integrity. All authors have read and agree to the manuscript as written.



A total of 4,270 patients underwent CABG, valve (aortic valve replacement, mitral valve replacement, or mitral valve repair) or CABG + valve at the QEII HSC from Jan 2004 to Dec 2009. The model cohort was 65% of the total case volume (6,525) performed during the study period. The distributions of risk factors in the development cohort are displayed in Table 2. The prevalence of MACE in this cohort was 15.7% (n=669). The prevalence of MACE was higher in the CABG plus valve group (32%, n=155) than in the isolated procedures (CABG 13%, n=416; Valve 14%, n=98). The frequencies of MACE for each procedure as well as the components of MACE are summarized in Table 3.

Table 2 Distribution of risk factors in the model development cohort
Table 3 Frequency of MACE and MACE components in the model development cohort

Model development

Assessment of the variance inflation for each variable revealed that no variable exceeded 4.0 allowing all variables to remain in the final model (see Additional file 1: Table 5).

By LOESS regression, the squared transformation of the continuous variable age had the most linear relationship with the logodds of the outcome (see Additonal file 1: Figure 2-3). Hemoglobin and creatinine had non-linear relationships with the outcome despite transformations. The inflection points of the natural variable were taken to create categorical variables. Hemoglobin inflection points were 115 and 135 and creatinine was 115, 140, and 160.

A total of 16 variables remain in the logistic regression model (Table 4). Significant predictors of MACE include variables such as Frailty, BMI >35, all levels of Creatinine, DM, Emergent and Urgent status and CABG + valve procedure type. Model beta coefficients are presented in the Additional file 1: Table 5.

Table 4 Logistic regression for MACE in a CABG/Valve population

Model performance

The concordance statistic for the logistic regression was 0.764, which is equivalent to an ROC of 76.4% (95% CI; 75–79). The Hosmer-Lemeshow goodness of fit statistic was not significant (p=0.3133).

The deciles of observed over predicted probabilities of MACE are plotted (Figure 1). Each data points falls on or very near the ideal line indicating excellent calibration.

Figure 1

Calibration plot of observed risk versus predicted risk.


This paper outlines the development of a logistic regression model. Our model is unique for three reasons: 1) it performs well in a heterogeneous population including CABG, valve, and CABG + valve patients, 2) it predicts a composite outcome of quality indicators including death and major morbidities and, 3) it was developed for a contemporary cohort that represents a contemporary cardiac surgery practice. Also, we could not identify in the literature any models that include a CABG, valve and valve + CABG population with a composite end point.

Although many models exist that describe isolated CABG or isolated valve [19, 1618], their use is limited to only a percentage of the cardiac surgical population. EuroSCORE, perhaps the most widely recognized cardiac surgical predictive model, is a mixed population model that has had great success in research and as a quality improvement tool [11, 18]. However, its derivation cohort is over 15 years old restricting its applicability to a contemporary practice. Our model is derived from a 2004–2009 cohort allowing for more current application. The EuroSCORE II has failed to perform better then the original model in certain procedure type subsets [13, 14]. Furthermore both EuroSCORE models only predicts mortality, and not other important quality indicators.

Provided the model performs well, the advantage of a heterogeneous population model exceeds that of single-procedure models in its ability to describe to majority of surgical patients and can help facilitate quality improvement efforts. However, certain predictive variables, such as descriptions of coronary artery disease or valve disease severity, cannot be included in the model, as they do not apply to the entire model derivation cohort. This might be problematic, as some variables like left main disease have previously been identified in the literature as highly important variables to include in CABG mortality models [25]. Although our model cannot contain this variable (as it would be entirely co-linear with CABG patients), it does contain other clinically relevant variables previously used in other published models [19, 1618].

Modeling composite outcomes allows for a broader prediction of important post-operative events rather than being limited to a single outcome. Also, the components of our composite are each identified as quality indicators in cardiac surgery [15] allowing for improved clinical validity. The benefit of a correctly chosen composite outcome allows for more detailed description and prediction of the clinical population increasing the clinical relevance of the model.

The model discrimination is high with a ROC 76.4% (75–79, 95% CI) that exceeds published recommendations [26] and is similar to other published cardiac surgical models [19, 1618]. This indicates that the model has good predictive ability. The bootstrap procedure, a form of internal validation [27, 28], allows for estimation of the 95% confidence interval. The tight 95% confidence interval provides a further estimation of reliability of the model. The calibration plot allows for a visual representation of the model’s performance (Figure 1). The observed to expected (O/E) data points fall on or very near the ideal line indicating excellent calibration of the model. The model performed well for each individual procedure subset as well (isolated CABG, isolated valve, CABG/valve).


We provide a statistically and clinically relevant model that is an essential tool in the era of quality improvement. The model includes heterogeneous procedures and a composite outcomes increasing its utility.

As the profile of the cardiac surgery patients changes, so must the predictive models used to describe this group. This is of utmost importance in the field of quality assessment and improvement. Models such as the one reported in this manuscript, assists many QI techniques. They can be used to perform pre-operative predictive risk matching to allow for comparison of matched groups and can risk adjust surgeon specific surgical outcomes for report carding [24, 26, 29]. The benefit of the mixed population and composite end points facilitates describing a contemporary clinical practice so that QI efforts are more productive. Developers of such models must be dedicated to upholding high statistical standards, so that the QI efforts actually benefit the patients. As QI efforts become a staple in cardiac surgery practice, models such as these are essential in propelling advancement in this field and improving outcomes for our patients.



Body mass index


Coronary artery bypass grafting


Confidence limits


Queen Elizabeth II health science center

QI Quality:



Locally weighted scatter plot smoothing


Major adverse cardiac events


New York heart association




Odds ratio


Variance inflation factor


World Health Organization.


  1. 1.

    Shahian DM, O'Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 1–coronary artery bypass grafting surgery. Ann Thorac Surg. 2009, 88: S2-S22. 10.1016/j.athoracsur.2009.05.053.

    Article  PubMed  Google Scholar 

  2. 2.

    Jones RH, Hannan EL, Hammermeister KE, Delong ER, O'Connor GT, Luepker RV: Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. The working group panel on the cooperative CABG database project. J Am Coll Cardiol. 1996, 28: 1478-1487. 10.1016/S0735-1097(96)00359-2.

    CAS  Article  PubMed  Google Scholar 

  3. 3.

    Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A: The society of thoracic surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg. 2003, 75: 1856-1864. 10.1016/S0003-4975(03)00179-6. discussion 1864–5

    Article  PubMed  Google Scholar 

  4. 4.

    O’Connor G, Plume S, Olmstead E, Coffin LH, Morton JR, Maloney CT: Multivariate prediction of in-hospital mortality associated with coronary artery bypass surgery. Northern new england cardiovascular disease study group. Circ. 1992, 85: 2110-2118. 10.1161/01.CIR.85.6.2110.

    Article  Google Scholar 

  5. 5.

    Shahian DM, O'Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 3–valve plus coronary artery bypass grafting surgery. Ann Thorac Surg. 2009, 88: S43-S62. 10.1016/j.athoracsur.2009.05.055.

    Article  PubMed  Google Scholar 

  6. 6.

    OBrien S, Shahian D, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 2--isolated valve surgery. Ann Thorac Surg. 2009, 88: S23-S42. 10.1016/j.athoracsur.2009.05.056.

    Article  Google Scholar 

  7. 7.

    Hannan EL, Racz MJ, Jones RH, Gold JP, Ryan TJ, Hafner JP: Predictors of mortality for patients undergoing cardiac valve replacements in new york state. Ann Thorac Surg. 2000, 70: 1212-1218. 10.1016/S0003-4975(00)01968-8.

    CAS  Article  PubMed  Google Scholar 

  8. 8.

    Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP: Risk index for predicting in-hospital mortality for cardiac valve surgery. Ann Thorac Surg. 2007, 83: 921-929. 10.1016/j.athoracsur.2006.09.051.

    Article  PubMed  Google Scholar 

  9. 9.

    Jamieson WR, Edwards FH, Schwartz M, Bero JW, Clark RE, Grover FL: Risk stratification for cardiac valve replacement. national cardiac surgery database. database committee of the society of thoracic surgeons. Ann Thorac Surg. 1999, 67: 943-951. 10.1016/S0003-4975(99)00175-7.

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    Nowicki ER, Birkmeyer NJ, Weintraub RW, Leavitt BJ, Sanders JH, Dacey LJ: Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in northern new england. Ann Thorac Surg. 2004, 77: 1966-1977. 10.1016/j.athoracsur.2003.12.035.

    Article  PubMed  Google Scholar 

  11. 11.

    Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E: Risk factors and outcome in european cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg. 1999, 15: 816-822. 10.1016/S1010-7940(99)00106-2. discussion 822–3

    CAS  Article  PubMed  Google Scholar 

  12. 12.

    Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, Lockowandt U: EuroSCORE II. Eur J Cardiothorac Surg. 2012, 41 (4): 734-744. 10.1093/ejcts/ezs043. Epub 2012 Feb 29

    Article  PubMed  Google Scholar 

  13. 13.

    Howell NJ, Head SJ, Freemantle N, van der Meulen TA, Senanayake E, Menon A, Kappetein AP, Pagano D: The new EuroSCORE II does not improve prediction of mortality in high-risk patients undergoing cardiac surgery: a collaborative analysis of two European centres. Eur J Cardiothorac Surg. 2013, Epub ahead of print

    Google Scholar 

  14. 14.

    Di Dedda U, Pelissero G, Agnelli B, De Vincentiis C, Castelvecchio S, Ranucci M: Accuracy, calibration and clinical performance of the new EuroSCORE II risk stratification system. Eur J Cardiothorac Surg. 2013, 43 (1): 27-32. 10.1093/ejcts/ezs196. Epub 2012 Jul 20

    Article  PubMed  Google Scholar 

  15. 15.

    Guru V, Anderson GM, Fremes SE, O'Connor GT, Grover FL, Tu JV: The identification and development of canadian coronary artery bypass graft surgery quality indicators. J Thorac Cardiovasc Surg. 2005, 130: 1257-

    Article  PubMed  Google Scholar 

  16. 16.

    Rankin JS, Hammill BG, Ferguson TB, Glower DD, O'Brien SM, DeLong ER: Determinants of operative mortality in valvular heart surgery. J Thorac Cardiovasc Surg. 2006, 131: 547-57. 10.1016/j.jtcvs.2005.10.041.

    Article  PubMed  Google Scholar 

  17. 17.

    Ambler G, Omar RZ, Royston P, Kinsman R, Keogh BE, Taylor KM: Generic, simple risk stratification model for heart valve surgery. Circulation. 2005, 112: 224-31. 10.1161/CIRCULATIONAHA.104.515049.

    Article  PubMed  Google Scholar 

  18. 18.

    Nashef SA, Roques F, Hammill BG, Peterson ED, Michel P, Grover FL: Validation of european system for cardiac operative risk evaluation (EuroSCORE) in north american cardiac surgery. Eur J Cardiothorac Surg. 2002, 22: 101-5. 10.1016/S1010-7940(02)00208-7.

    Article  PubMed  Google Scholar 

  19. 19.

    Cleveland WS: Robust locally weighted regression and smoothing scatter- plots. J Amer Stats Ass. 1979, 74: 829-36. 10.1080/01621459.1979.10481038.

    Article  Google Scholar 

  20. 20.

    WHO: WHO Technical Report Series 854. Physical status: the use and interpretation of anthropometry. Report of a WHO Expert Committee. 1995, Geneva: World Health Organization

    Google Scholar 

  21. 21.

    Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982, 143: 29-36.

    CAS  Article  PubMed  Google Scholar 

  22. 22.

    Hosmer DW, Lemeshow S: A goodness-of-fit test for the multiple regression mdel. Commun Statistics. 1980, A10: 1043-1069.

    Article  Google Scholar 

  23. 23.

    Marcin JP, Romano PS: Size matters to a model's fit. Crit Care Med. 2007, 35: 2212-3. 10.1097/01.CCM.0000281522.70992.EF.

    Article  PubMed  Google Scholar 

  24. 24.

    Harrell FE, Lee KL, Mark DB: Tutorial in biostatistics multivariable prognostic models: issues in developing mdels, evaluating assumptions and adequacy, and measuring and reducing errors. Stats Med. 1996, 15: 361-87. 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.

    Article  Google Scholar 

  25. 25.

    Tu JV, Sykora K, Naylor CD: Assessing the outcomes of coronary artery bypass graft surgery: how many risk factors are enough? steering committee of the cardiac care network of ontario. J Am Coll Cardiol. 1997, 30: 1317-23. 10.1016/S0735-1097(97)00295-7.

    CAS  Article  PubMed  Google Scholar 

  26. 26.

    Spiegelhalter DJ: Probabilistic prediction in patient management and clinical trials. Stat Med. 1986, 5: 421-33. 10.1002/sim.4780050506.

    CAS  Article  PubMed  Google Scholar 

  27. 27.

    Concato J, Feinstein AR, Holford TR: The risk of determining risk with multivariable models. Ann Intern Med. 1993, 118: 201-10. 10.7326/0003-4819-118-3-199302010-00009.

    CAS  Article  PubMed  Google Scholar 

  28. 28.

    Altman DG, Royston P: What do we mean by validating a prognostic model?. Stat Med. 2000, 19: 453-73. 10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5.

    CAS  Article  PubMed  Google Scholar 

  29. 29.

    Harrell F, Califf R, Pryor D, Lee K, Rosati R: Evaluating the yield of medical tests. JAMA. 1982, 247: 2543-6. 10.1001/jama.1982.03320430047030.

    Article  PubMed  Google Scholar 

Download references


This work was supported by the Heart and Stroke Foundation of Canada grant-in-aid and a Nova Scotia Health Research Foundation salary grant (PSO-SRA-2010–6546).

Author information



Corresponding author

Correspondence to Christine R Herman.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

CRH carried out the data analysis and authored the manuscript. KJB provided statistical support. CRH, KJB and RB participated in the design of the study. All authors contributed to the editing of the manuscript and have approved the final manuscript.

Electronic supplementary material

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Herman, C.R., Buth, K.J., Légaré, J. et al. Development of a predictive model for major adverse cardiac events in a coronary artery bypass and valve population. J Cardiothorac Surg 8, 177 (2013).

Download citation


  • Cardiac surgery
  • Predictive model
  • Outcomes