Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Development of a predictive model for major adverse cardiac events in a coronary artery bypass and valve population

  • Christine R Herman1, 2Email author,
  • Karen J Buth1,
  • Jean-François Légaré1,
  • Adrian R Levy2 and
  • Roger Baskett1
Journal of Cardiothoracic Surgery20138:177

https://doi.org/10.1186/1749-8090-8-177

Received: 24 August 2012

Accepted: 6 June 2013

Published: 30 July 2013

Abstract

Background

Quality improvement initiatives in cardiac surgery largely rely on risk prediction models. Most often, these models include isolated populations and describe isolated end-points. However, with the changing clinical profile of the cardiac surgical patients, mixed populations models are required to accurately represent the majority of the surgical population. Also, composite model end-points of morbidity and mortality, better reflect outcomes experienced by patients.

Methods

The model development cohort included 4,270 patients who underwent aortic or mitral valve replacement, or mitral valve repair with/without coronary artery bypass grafting, or isolated coronary artery bypass grafting. A composite end-point of infection, stroke, acute renal failure, or death was evaluated. Age, sex, surgical priority, and procedure were forced, a priori, into the model and then stepwise selection of candidate variables was utilized. Model performance was evaluated by concordance statistic, Hosmer-Lemeshow Goodness of Fit, and calibration plots. Bootstrap technique was employed to validate the model.

Results

The model included 16 variables. Several variables were significant such as, emergent surgical priority (OR 4.3; 95% CI 2.9-7.4), CABG + Valve procedure (OR 2.3; 95% CI 1.8-3.0), and frailty (OR 1.7; 95% CI 1.2-2.5), among others. The concordance statistic for the major adverse cardiac events model in a mixed population was 0.764 (95% CL; 0.75-0.79) and had excellent calibration.

Conclusions

Development of predictive models with composite end-points and mixed procedure population can yield robust statistical and clinical validity. As they more accurately reflect current cardiac surgical profile, models such as this, are an essential tool in quality improvement efforts.

Keywords

Cardiac surgery Predictive model Outcomes

Background

Quality improvement initiatives (QI), a cornerstone of cardiac surgery, have largely relied on predictive models to advance the quality of care cardiac surgical patients receive. For the last two decades, coronary artery bypass grafting (CABG) has dominated clinical practice in cardiac surgery, and therefore the majority of quality improvement initiatives have focused on surgical outcomes following isolated CABG surgery [14]. More recently, however, there has been an increase in valve and valve-CABG cases in cardiac surgery [5] and existing predictive models for isolated CABG may not accurately reflect current practice profiles. These models may achieve great statistical validity but lack clinical validity as the models are applicable to only a decreasing percentage of the clinical practice. In order to maintain continued success in quality improvement, it is important to delineate risk profiles for a group of mixed procedures, including CABG, valve, and valve + CABG, that characterize current clinical practice.

To accommodate the shift in the profile of cardiac surgical practice, isolated valve and valve plus CABG models have been developed [510]. Some debate exists regarding the validity of developing models with hetero-geneous procedures with some advocating for single procedure models [7, 9, 10]. Homogeneity within the population allows for simplicity of model development with improved reliability but limits sample size available for development and validation [5]. Although mixed procedural models may be confounded or biased by different pathophysiological and risk profiles, they improve sample size [5] and increase their relevance to current surgical practice.

Largely, mixed procedural models have focused on aortic/mitral or valve +/− CABG, with few CABG, valve and CABG +valve models present in the literature. The EuroSCORE model, one of the mostly widely used cardiac surgical prognostic models, has successfully achieved both statistical and clinical relevance as it applies to a mixed surgical practice including CABG, valve, and CABG + valve. However its development and validation cohort were derived from a population sample that is over fifteen years old [11] limiting its applicability to a contemporary surgical practice. Recently, the EuroSCORE II updated its risk model obviating some of the limitations of the original model [12]. However, the performance of this heterogeneous procedure model performed poorly in risk prediction for several of its component procedures types [13, 14]. In addition, both the EuroSCORE and the EuroSCORE II are only a mortality model.

Many models describe death as a lone end point, despite that fact that cardiac patients experience a variety of relevant morbidity. These alternate surgical outcomes are also important quality indicators for cardiac surgical care [15] and targets for QI initiatives. Subsequently, modeling composite end points such as, major adverse cardiac events (MACE), that included both important morbidity and mortality may provide more insight to the surgical outcomes experienced by patients as well as increase statistical power in low frequency end points.

Comprehensive models that include most major cardiac surgical procedures may allow for better understanding of patient risk profiles and facilitate quality improvement initiatives directed towards the majority of patients seen in current practice. The objective of this paper is to develop a CABG, valve, CABG + valve morbidity and mortality (MACE) model that can be applied to the majority of patients undergoing cardiac surgery.

Methods

Data source and study population

This study is a retrospective cohort design. The Maritime Heart Center Cardiac Surgery Registry is a detailed clinical database housed at the Queen Elizabeth II Health Science Center (QEII HSC) Halifax, Nova Scotia, an academic tertiary care centre performing 1200 cardiac cases per year. It includes pre-, intra-, and post-operative data prospectively collected on all cardiac surgical cases performed at the QEII HSC from 1995 to present. Trained abstractors collect data, and a database administrator maintains the registry. The database is audited annually.

The model development cohort included all patients undergoing isolated coronary artery bypass grafting (CABG), isolated aortic valve replacement, isolated mitral valve repair or replacement with or without concomitant CABG performed at the QEII HSC since 2004 to 2009. The development cohort was restricted to these years in order to maintain a modern population relevant to current clinical practice.

The primary outcome was a composite end point defined as Major Adverse Cardiac Events (MACE) that included in-hospital death, stroke (persisting at discharge or transient), acute renal failure (new post-operative renal failure or acute on chronic (>50% increase from baseline creatinine)), or infection (sepsis, pneumonia, or deep sternal wound infection).

Statistical methods

Variable selection

Candidate variables for model development included the following preoperative characteristics: age, sex, diabetes, frailty, chronic obstructive pulmonary disease, redo sternotomy, atrial fibrillation, hemoglobin, peripheral vascular disease, cerebro-vascular disease, creatinine, ejection fraction < 40%, New York Heart Association classification (NYHA I-IV) and body mass index (BMI). Surgery-related data such as urgency of surgery, and procedure type were also included. These candidate variables were chosen a priori and selected through rigorous review of the literature [19, 1618] (Table 1).
Table 1

Model variable definitions

Variable

Definition

Age

Patient age at the time of surgery

Gender

Male or Female

Body mass index

Calculated in kilograms and centimeters.

Diabetes

Any history of Diabetes Mellitus, regardless of duration

Pre-op Afib

Any previously documented history of Atrial Fibrillation

COPD

Any preivous documented history of Chronic Obstructive Pulmonary Disease

CVD

Any Transient Ischemic Attack, Cerebrovascular Accident/Stroke, history of cerebrovascular surgery, or any carotid disease.

PVD

Whether the patient has Peripheral Vascular Disease, as indicated by claudication; amputation for arterial insufficiency; aorto-iliac occlusive disease reconstruction; peripheral vascular bypass surgery, angioplasty, or stent; documented AAA.

Frailty

Any deficiency in the Katz index of Activities of Daily Living (independence in feeding, bathing, dressing, transferring, toileting, and urinary continence), as well as independence in ambulation (no walking aid or assist required) or any clear evidence of a previous diagnosis of dementia by a physician.

EF<40

Ejection fraction measured less than 40% by any modality.

NYHA (I-IV)

New York Heart Association Class. I = Patients with cardiac disease but without limitation of physical activity. II = Patients with cardiac disease resulting in slight limitation of physical activity (fatigue, palpitations, dyspnea, or anginal pain). III = Patients with cardiac disease resulting in marked limitation of physical activity. IV = Patients with cardiac disease resulting in inability to carry on any physical activity without discomfort. Symptoms of cardiac insufficiency or of the anginal syndrome may be present even at rest.

Hemoglobin

Most recent hemoglobin level prior to day of surgery.

Pre-op creatinine

Highest preop serum creatinine for this admission.

Redo sternotomy

Any history of previous surgery that traversed the anterior mediastinum.

Surigical priority

Elective [stable at home], in-house [requiring hospitalization until the time of surgery], urgent [requiring surgery within 24 hours to minimize further clinical deterioration], or emergent [no delay in surgery].

Procedure

Any Coronary artery bypass grafting, aortic valve replacement or repair with/without CABG, or Mitral valve replacement or repair with/without CABG.

Multi-colinearity of candidate variables was assessed with variance inflation factor (VIF) (Additional file 1: see Statistical Methods). The linear relationships of the natural variables and their transformations were assessed through locally weighted scatterplot smoothing (LOESS) regression [19] (see Additional file 1: Linearity and Transformations and Figure 2-3). The WHO classification of BMI was used [20].

Model evaluation

A multivariate logistic regression analysis was used to describe MACE.

Age, sex, procedure type and surgical priority were chosen a priori and forced into the model. Stepwise selection was implemented for the remainder of the candidate variables. The concordance statistic and −2 log likelihood were evaluated to assess the contribution of each variable to the model. If a variable did not contribute to an increase in the C or −2 log likelihood statistic it was not retained in the model.

Model discrimination was determined using the concordance statistic [21]. Model calibration was assessed by the Hosmer-Lemeshow goodness-of-fit statistic [22, 23] as well as calibration plots [24]. Deciles of observed and predicted probabilities of MACE were plotted for the calibration plots [5]. Bootstrap procedure was used to internally validate the model.

All statistical analysis was performed using SAS software version 9.2 (SAS, Cary, NC).

Approval for conducting this study was obtained from the Institutional Review Board of the Capital District Health Authority. The requirement to obtain informed consent was waived under Section 2.1c of the Tri-Council Policy Statement. All personal identifiers were stripped prior to data analysis to ensure patient anonymity and confidentiality.

The authors had full access to the data and take full responsibility for its integrity. All authors have read and agree to the manuscript as written.

Results

Population

A total of 4,270 patients underwent CABG, valve (aortic valve replacement, mitral valve replacement, or mitral valve repair) or CABG + valve at the QEII HSC from Jan 2004 to Dec 2009. The model cohort was 65% of the total case volume (6,525) performed during the study period. The distributions of risk factors in the development cohort are displayed in Table 2. The prevalence of MACE in this cohort was 15.7% (n=669). The prevalence of MACE was higher in the CABG plus valve group (32%, n=155) than in the isolated procedures (CABG 13%, n=416; Valve 14%, n=98). The frequencies of MACE for each procedure as well as the components of MACE are summarized in Table 3.
Table 2

Distribution of risk factors in the model development cohort

Variable

 

All Procedures

CABG

AVR/MVR/MVrpr

CABG + Valve

  

n=4270

n=3095

n=696

n=479

  

(%)

(%)

(%)

(%)

Age

 

67 (IQR 59–74)

66 (IQR 58–73)

67 (IQR 57–75)

74 (IQR 67–80)

Female

 

25

21

39

31

BMI (kg/m2)

<25

22

20

28

26

 

25-30

40

40

41

43

 

30-35

25

27

18

23

 

>35

13

13

13

8

Diabetes

 

36

40

23

34

Pre-op Afib

 

12

9

21

22

COPD

 

15

14

18

17

CVD

 

14

14

13

17

PVD

 

17

19

7

18

Frailty

 

4

3

6

8

EF<40

 

15

15

11

20

NYHA

I

35

44

13

13

 

II

23

23

29

21

 

III

26

20

42

42

 

IV

15

13

17

24

HGB (g/L)

<115

50

52

49

38

 

115-135

33

32

32

41

 

>135

17

15

19

21

Pre-op creatinine

<115

74

76

78

62

(μmol/L)

115-140

14

13

12

18

 

140-160

5

5

3

8

 

>160

7

7

7

11

Redo sternotomy

 

7

4

19

11

Status

Elective

47

44

59

44

 

In-house

41

41

36

43

 

Urgent

9

11

4

10

 

Emergent

3

4

2

3

BMI Body mass index, HGB Hemoglobin, Afib Atrial Fibrillation, COPD Chronic Obstructive Pulmonary Disease, CVD Cerebro Vascular Disease, PVD Peripheral Vascular Disease, EF Ejection Fraction, NYHA New York Heart Association.

Table 3

Frequency of MACE and MACE components in the model development cohort

Variable

 

All Procedures

CABG

AVR/MVR/MVrpr

CABG + Valve

  

n=4270

n=3095

n=696

n=479

  

(%)

(%)

(%)

(%)

MACE*

 

15.7

13.0

14.0

32.0

Mortality

 

4.2

3.3

3.3

11.5

Acute renal failure

 

6.5

5.5

5.5

14.0

Any stroke

 

2.9

2.3

2.4

7.3

 

Transient (<24 h)

1.7

0.9

1.2

3.1

 

Permanent

1.7

1.4

1.3

4.2

Infection

 

8.0

7.1

6.3

16.1

 

Deep sternal Wound infection

1.1

1.0

0.1

2.9

 

Sepsis

2.8

2.4

1.9

7.3

 

Pneumonia

6.5

5.7

5.3

13.6

*MACE—Major Adverse Cardiac Event defined as in-hospital death, stroke (persisting at discharge or transient), acute renal failure (new post-operative renal failure or acute on chronic (>50% increase from baseline creatinine)), or infection (sepsis, pneumonia, or deep sternal wound infection).

Model development

Assessment of the variance inflation for each variable revealed that no variable exceeded 4.0 allowing all variables to remain in the final model (see Additional file 1: Table 5).

By LOESS regression, the squared transformation of the continuous variable age had the most linear relationship with the logodds of the outcome (see Additonal file 1: Figure 2-3). Hemoglobin and creatinine had non-linear relationships with the outcome despite transformations. The inflection points of the natural variable were taken to create categorical variables. Hemoglobin inflection points were 115 and 135 and creatinine was 115, 140, and 160.

A total of 16 variables remain in the logistic regression model (Table 4). Significant predictors of MACE include variables such as Frailty, BMI >35, all levels of Creatinine, DM, Emergent and Urgent status and CABG + valve procedure type. Model beta coefficients are presented in the Additional file 1: Table 5.
Table 4

Logistic regression for MACE in a CABG/Valve population

Variable

 

Odds ratio

95% confidence limit

Age square

 

1.025

1.02-1.03

Female

 

0.8

0.6-1.0

PVD

 

1.6

1.2-1.9

Frailty

 

1.7

1.2-2.5

BMI

25-30

1.0

-

<25

1.2

0.9-1.5

30-35

1.2

0.9-1.5

>35

1.5

1.1-2.1

NYHA I

 

1.0

-

NYHA II

 

1.2

0.9-1.6

NYHA III

 

1.2

0.9-1.5

NYHA IV

 

1.3

1.0-1.8

HGB <115

 

1.2

1.0-1.5

HGB 115-135

 

1.8

1.3-2.3

HGB >135

 

1.0

-

Creatinine

<115

1.0

-

115-140

1.3

1.1-1.7

140-160

1.6

1.1-2.2

>160

1.6

1.2-2.2

Preop Afib

 

1.4

1.1-1.8

Diabetes

 

1.5

1.1-1.8

EF < 40

 

1.3

1.0-1.7

COPD

 

1.2

0.9-1.5

CVD

 

1.2

1.0-1.6

HTN

 

1.1

0.9-1.5

Preop RF

 

1.3

0.9-1.9

Procedure

CABG

1.0

-

Valve

1.2

0.9-1.5

CABG + Valve

2.3

1.8-3.0

Surgical priority elective

 

1.0

-

In-house

1.1

0.9-1.5

Urgent

1.8

1.8-3.6

Emergent

4.3

2.9-7.4

Redo

 

1.4

0.9-1.9

Model performance

The concordance statistic for the logistic regression was 0.764, which is equivalent to an ROC of 76.4% (95% CI; 75–79). The Hosmer-Lemeshow goodness of fit statistic was not significant (p=0.3133).

The deciles of observed over predicted probabilities of MACE are plotted (Figure 1). Each data points falls on or very near the ideal line indicating excellent calibration.
Figure 1

Calibration plot of observed risk versus predicted risk.

Discussion

This paper outlines the development of a logistic regression model. Our model is unique for three reasons: 1) it performs well in a heterogeneous population including CABG, valve, and CABG + valve patients, 2) it predicts a composite outcome of quality indicators including death and major morbidities and, 3) it was developed for a contemporary cohort that represents a contemporary cardiac surgery practice. Also, we could not identify in the literature any models that include a CABG, valve and valve + CABG population with a composite end point.

Although many models exist that describe isolated CABG or isolated valve [19, 1618], their use is limited to only a percentage of the cardiac surgical population. EuroSCORE, perhaps the most widely recognized cardiac surgical predictive model, is a mixed population model that has had great success in research and as a quality improvement tool [11, 18]. However, its derivation cohort is over 15 years old restricting its applicability to a contemporary practice. Our model is derived from a 2004–2009 cohort allowing for more current application. The EuroSCORE II has failed to perform better then the original model in certain procedure type subsets [13, 14]. Furthermore both EuroSCORE models only predicts mortality, and not other important quality indicators.

Provided the model performs well, the advantage of a heterogeneous population model exceeds that of single-procedure models in its ability to describe to majority of surgical patients and can help facilitate quality improvement efforts. However, certain predictive variables, such as descriptions of coronary artery disease or valve disease severity, cannot be included in the model, as they do not apply to the entire model derivation cohort. This might be problematic, as some variables like left main disease have previously been identified in the literature as highly important variables to include in CABG mortality models [25]. Although our model cannot contain this variable (as it would be entirely co-linear with CABG patients), it does contain other clinically relevant variables previously used in other published models [19, 1618].

Modeling composite outcomes allows for a broader prediction of important post-operative events rather than being limited to a single outcome. Also, the components of our composite are each identified as quality indicators in cardiac surgery [15] allowing for improved clinical validity. The benefit of a correctly chosen composite outcome allows for more detailed description and prediction of the clinical population increasing the clinical relevance of the model.

The model discrimination is high with a ROC 76.4% (75–79, 95% CI) that exceeds published recommendations [26] and is similar to other published cardiac surgical models [19, 1618]. This indicates that the model has good predictive ability. The bootstrap procedure, a form of internal validation [27, 28], allows for estimation of the 95% confidence interval. The tight 95% confidence interval provides a further estimation of reliability of the model. The calibration plot allows for a visual representation of the model’s performance (Figure 1). The observed to expected (O/E) data points fall on or very near the ideal line indicating excellent calibration of the model. The model performed well for each individual procedure subset as well (isolated CABG, isolated valve, CABG/valve).

Conclusions

We provide a statistically and clinically relevant model that is an essential tool in the era of quality improvement. The model includes heterogeneous procedures and a composite outcomes increasing its utility.

As the profile of the cardiac surgery patients changes, so must the predictive models used to describe this group. This is of utmost importance in the field of quality assessment and improvement. Models such as the one reported in this manuscript, assists many QI techniques. They can be used to perform pre-operative predictive risk matching to allow for comparison of matched groups and can risk adjust surgeon specific surgical outcomes for report carding [24, 26, 29]. The benefit of the mixed population and composite end points facilitates describing a contemporary clinical practice so that QI efforts are more productive. Developers of such models must be dedicated to upholding high statistical standards, so that the QI efforts actually benefit the patients. As QI efforts become a staple in cardiac surgery practice, models such as these are essential in propelling advancement in this field and improving outcomes for our patients.

Abbreviations

BMI: 

Body mass index

CABG: 

Coronary artery bypass grafting

CL: 

Confidence limits

QEII HSC: 

Queen Elizabeth II health science center

QI Quality: 

Improvement

LOESS: 

Locally weighted scatter plot smoothing

MACE: 

Major adverse cardiac events

NYHA: 

New York heart association

O/E: 

Observed/Expected

OR: 

Odds ratio

VIF: 

Variance inflation factor

WHO: 

World Health Organization.

Declarations

Acknowledgments

This work was supported by the Heart and Stroke Foundation of Canada grant-in-aid and a Nova Scotia Health Research Foundation salary grant (PSO-SRA-2010–6546).

Authors’ Affiliations

(1)
Division of Cardiac Surgery, Queen Elizabeth II Health Science Center
(2)
Department of Community Health and Epidemiology, Dalhousie University

References

  1. Shahian DM, O'Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 1–coronary artery bypass grafting surgery. Ann Thorac Surg. 2009, 88: S2-S22. 10.1016/j.athoracsur.2009.05.053.View ArticlePubMedGoogle Scholar
  2. Jones RH, Hannan EL, Hammermeister KE, Delong ER, O'Connor GT, Luepker RV: Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. The working group panel on the cooperative CABG database project. J Am Coll Cardiol. 1996, 28: 1478-1487. 10.1016/S0735-1097(96)00359-2.View ArticlePubMedGoogle Scholar
  3. Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A: The society of thoracic surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg. 2003, 75: 1856-1864. 10.1016/S0003-4975(03)00179-6. discussion 1864–5View ArticlePubMedGoogle Scholar
  4. O’Connor G, Plume S, Olmstead E, Coffin LH, Morton JR, Maloney CT: Multivariate prediction of in-hospital mortality associated with coronary artery bypass surgery. Northern new england cardiovascular disease study group. Circ. 1992, 85: 2110-2118. 10.1161/01.CIR.85.6.2110.View ArticleGoogle Scholar
  5. Shahian DM, O'Brien SM, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 3–valve plus coronary artery bypass grafting surgery. Ann Thorac Surg. 2009, 88: S43-S62. 10.1016/j.athoracsur.2009.05.055.View ArticlePubMedGoogle Scholar
  6. OBrien S, Shahian D, Filardo G, Ferraris VA, Haan CK, Rich JB: The society of thoracic surgeons 2008 cardiac surgery risk models: part 2--isolated valve surgery. Ann Thorac Surg. 2009, 88: S23-S42. 10.1016/j.athoracsur.2009.05.056.View ArticleGoogle Scholar
  7. Hannan EL, Racz MJ, Jones RH, Gold JP, Ryan TJ, Hafner JP: Predictors of mortality for patients undergoing cardiac valve replacements in new york state. Ann Thorac Surg. 2000, 70: 1212-1218. 10.1016/S0003-4975(00)01968-8.View ArticlePubMedGoogle Scholar
  8. Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP: Risk index for predicting in-hospital mortality for cardiac valve surgery. Ann Thorac Surg. 2007, 83: 921-929. 10.1016/j.athoracsur.2006.09.051.View ArticlePubMedGoogle Scholar
  9. Jamieson WR, Edwards FH, Schwartz M, Bero JW, Clark RE, Grover FL: Risk stratification for cardiac valve replacement. national cardiac surgery database. database committee of the society of thoracic surgeons. Ann Thorac Surg. 1999, 67: 943-951. 10.1016/S0003-4975(99)00175-7.View ArticlePubMedGoogle Scholar
  10. Nowicki ER, Birkmeyer NJ, Weintraub RW, Leavitt BJ, Sanders JH, Dacey LJ: Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in northern new england. Ann Thorac Surg. 2004, 77: 1966-1977. 10.1016/j.athoracsur.2003.12.035.View ArticlePubMedGoogle Scholar
  11. Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E: Risk factors and outcome in european cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg. 1999, 15: 816-822. 10.1016/S1010-7940(99)00106-2. discussion 822–3View ArticlePubMedGoogle Scholar
  12. Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, Lockowandt U: EuroSCORE II. Eur J Cardiothorac Surg. 2012, 41 (4): 734-744. 10.1093/ejcts/ezs043. Epub 2012 Feb 29View ArticlePubMedGoogle Scholar
  13. Howell NJ, Head SJ, Freemantle N, van der Meulen TA, Senanayake E, Menon A, Kappetein AP, Pagano D: The new EuroSCORE II does not improve prediction of mortality in high-risk patients undergoing cardiac surgery: a collaborative analysis of two European centres. Eur J Cardiothorac Surg. 2013, Epub ahead of printGoogle Scholar
  14. Di Dedda U, Pelissero G, Agnelli B, De Vincentiis C, Castelvecchio S, Ranucci M: Accuracy, calibration and clinical performance of the new EuroSCORE II risk stratification system. Eur J Cardiothorac Surg. 2013, 43 (1): 27-32. 10.1093/ejcts/ezs196. Epub 2012 Jul 20View ArticlePubMedGoogle Scholar
  15. Guru V, Anderson GM, Fremes SE, O'Connor GT, Grover FL, Tu JV: The identification and development of canadian coronary artery bypass graft surgery quality indicators. J Thorac Cardiovasc Surg. 2005, 130: 1257-View ArticlePubMedGoogle Scholar
  16. Rankin JS, Hammill BG, Ferguson TB, Glower DD, O'Brien SM, DeLong ER: Determinants of operative mortality in valvular heart surgery. J Thorac Cardiovasc Surg. 2006, 131: 547-57. 10.1016/j.jtcvs.2005.10.041.View ArticlePubMedGoogle Scholar
  17. Ambler G, Omar RZ, Royston P, Kinsman R, Keogh BE, Taylor KM: Generic, simple risk stratification model for heart valve surgery. Circulation. 2005, 112: 224-31. 10.1161/CIRCULATIONAHA.104.515049.View ArticlePubMedGoogle Scholar
  18. Nashef SA, Roques F, Hammill BG, Peterson ED, Michel P, Grover FL: Validation of european system for cardiac operative risk evaluation (EuroSCORE) in north american cardiac surgery. Eur J Cardiothorac Surg. 2002, 22: 101-5. 10.1016/S1010-7940(02)00208-7.View ArticlePubMedGoogle Scholar
  19. Cleveland WS: Robust locally weighted regression and smoothing scatter- plots. J Amer Stats Ass. 1979, 74: 829-36. 10.1080/01621459.1979.10481038.View ArticleGoogle Scholar
  20. WHO: WHO Technical Report Series 854. Physical status: the use and interpretation of anthropometry. Report of a WHO Expert Committee. 1995, Geneva: World Health OrganizationGoogle Scholar
  21. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982, 143: 29-36.View ArticlePubMedGoogle Scholar
  22. Hosmer DW, Lemeshow S: A goodness-of-fit test for the multiple regression mdel. Commun Statistics. 1980, A10: 1043-1069.View ArticleGoogle Scholar
  23. Marcin JP, Romano PS: Size matters to a model's fit. Crit Care Med. 2007, 35: 2212-3. 10.1097/01.CCM.0000281522.70992.EF.View ArticlePubMedGoogle Scholar
  24. Harrell FE, Lee KL, Mark DB: Tutorial in biostatistics multivariable prognostic models: issues in developing mdels, evaluating assumptions and adequacy, and measuring and reducing errors. Stats Med. 1996, 15: 361-87. 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.View ArticleGoogle Scholar
  25. Tu JV, Sykora K, Naylor CD: Assessing the outcomes of coronary artery bypass graft surgery: how many risk factors are enough? steering committee of the cardiac care network of ontario. J Am Coll Cardiol. 1997, 30: 1317-23. 10.1016/S0735-1097(97)00295-7.View ArticlePubMedGoogle Scholar
  26. Spiegelhalter DJ: Probabilistic prediction in patient management and clinical trials. Stat Med. 1986, 5: 421-33. 10.1002/sim.4780050506.View ArticlePubMedGoogle Scholar
  27. Concato J, Feinstein AR, Holford TR: The risk of determining risk with multivariable models. Ann Intern Med. 1993, 118: 201-10. 10.7326/0003-4819-118-3-199302010-00009.View ArticlePubMedGoogle Scholar
  28. Altman DG, Royston P: What do we mean by validating a prognostic model?. Stat Med. 2000, 19: 453-73. 10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5.View ArticlePubMedGoogle Scholar
  29. Harrell F, Califf R, Pryor D, Lee K, Rosati R: Evaluating the yield of medical tests. JAMA. 1982, 247: 2543-6. 10.1001/jama.1982.03320430047030.View ArticlePubMedGoogle Scholar

Copyright

© Herman et al.; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement