Development and Initial Validation of Tool to Predict HF
Development and Initial Validation of Tool to Predict HF
The derivation data set identified four independent clinical predictors of heart failure—gender; previous history of myocardial infarction; lung crepitations; and ankle oedema. Table 1 shows the prevalence of these clinical features in the data sets utilized in the model derivation and validation. The mean age of subjects in the derivation data set was 71 years, 34% had a diagnosis of heart failure, and 29% were classified as New York Heart Association (NYHA) III or IV. The validation data sets mean age ranged from 66 to 76 years, heart failure prevalence from 13% to 29%, and those with NYHA class III or IV from 24% to 49%, but was unavailable for two studies.
The sensitivity and specificity of each of the clinical features, an abnormal ECG, and the natriuretic peptides are given in Supplementarymaterial,Table S2. The individual clinical features were less informative than both ECG and natriuretic peptide in all data sets except Cost.
Table 2 shows the results of univariable and multivariable logistic models for predicting heart failure that were tested in the derivation data set. These involved clinical features alone (Model 1), natriuretic peptides alone (Models 2 and 3: BNP or NT-proBNP), ECG alone (Model 4), and different combinations (Models 5–9). No significant interactions were found between age and gender, or between natriuretic peptide and the pre-specified list of patient characteristics. Ankle oedema was not a statistically significant predictor in all models but has remained to allow comparison between models. Except for gender, the diagnostic contributions of clinical and ECG items are reduced when used in combination with natriuretic peptides.
A model comprising clinical features plus natriuretic peptides was identified as the most parsimonious model (Model 5 or Model 6). The AUROCs and deviances of the models are available online (Supplementary material, Results I, Table S3).
External validation of Models 5 and 6 found that AUROCs were similar in the validation data sets to those in the derivation data set, with AUROCs varying between 0.84 and 0.93 (see Supplementary material,Table S4).
The calibration plots of Models 5 and 6 for the derivation and validation data sets are shown in Figure 1. The post-test probabilities of the derivation data set fall reasonably close to the diagonal line, indicating reasonable calibration. The NT-proBNP with clinical features model (Model 6) showed good calibration when applied to each validation data set, across the full range of the probability scale. The BNP with clinical features model (Model 5) was less well calibrated, though the 95% confidence intervals crossed the diagonal line in all cases except one (Cost ), when predicted risk of heart failure was between 0.2 and 0.4.
(Enlarge Image)
Figure 1.
Calibration plots for the clinical and natriuretic peptide model. The circles represent the observed proportion that had heart failure in the data set, and the vertical lines the 95% confidence interval around this proportion.
(Enlarge Image)
Figure 2.
The MICE rule.
The model was then split into a two-stage process: clinical features alone, and clinical features plus natriuretic peptide result. The Supplementary material, Figure S1 gives estimates of likelihood ratios and post-test probabilities for the combinations of clinical features alone, and shows that if the pre-test probability is set at 25%, the probability threshold of 20% is exceeded for all combinations except males and females without any of the features, and in females with ankle oedema. Increasing the pre-test probability to 35% made little difference to this result (though females with ankle oedema just crossed the threshold with a post-test probability of 21%).
This suggested a simple clinical rule abbreviated as MICE: Male, Infarction, Crepitations, Edema.
For the three groups recommended for further natriuretic peptide testing, test thresholds are shown in Table 3 at which the post-test probability of heart failure exceeds 20, 25, and 30%, representing different possible referral thresholds for echocardiography.
The performance of the one-stage model (clinical features plus natriuretic peptides) and the two-stage model (clinical features then natriuretic peptides as used in the MICE rule) in terms of sensitivity, specificity, and positive and negative predictive values is shown in Table 4. Sensitivity of the one-stage model using BNP varied in the different data sets between 67% and 86%, and using NT-proBNP between 77% and 92%. Sensitivity of the two-stage model varied between 81% and 89% for BNP and between 90% and 96% for NT-proBNP. Specificity of the one-stage model for BNP varied between 62% and 86%, and for NT-proBNP between 66% and 80%. Specificity of the two-stage model varied between 57% and 74% for BNP and 58% and 63% for NT-proBNP.
Results
The derivation data set identified four independent clinical predictors of heart failure—gender; previous history of myocardial infarction; lung crepitations; and ankle oedema. Table 1 shows the prevalence of these clinical features in the data sets utilized in the model derivation and validation. The mean age of subjects in the derivation data set was 71 years, 34% had a diagnosis of heart failure, and 29% were classified as New York Heart Association (NYHA) III or IV. The validation data sets mean age ranged from 66 to 76 years, heart failure prevalence from 13% to 29%, and those with NYHA class III or IV from 24% to 49%, but was unavailable for two studies.
The sensitivity and specificity of each of the clinical features, an abnormal ECG, and the natriuretic peptides are given in Supplementarymaterial,Table S2. The individual clinical features were less informative than both ECG and natriuretic peptide in all data sets except Cost.
Table 2 shows the results of univariable and multivariable logistic models for predicting heart failure that were tested in the derivation data set. These involved clinical features alone (Model 1), natriuretic peptides alone (Models 2 and 3: BNP or NT-proBNP), ECG alone (Model 4), and different combinations (Models 5–9). No significant interactions were found between age and gender, or between natriuretic peptide and the pre-specified list of patient characteristics. Ankle oedema was not a statistically significant predictor in all models but has remained to allow comparison between models. Except for gender, the diagnostic contributions of clinical and ECG items are reduced when used in combination with natriuretic peptides.
Selection of the Parsimonious Model
A model comprising clinical features plus natriuretic peptides was identified as the most parsimonious model (Model 5 or Model 6). The AUROCs and deviances of the models are available online (Supplementary material, Results I, Table S3).
Validation
External validation of Models 5 and 6 found that AUROCs were similar in the validation data sets to those in the derivation data set, with AUROCs varying between 0.84 and 0.93 (see Supplementary material,Table S4).
The calibration plots of Models 5 and 6 for the derivation and validation data sets are shown in Figure 1. The post-test probabilities of the derivation data set fall reasonably close to the diagonal line, indicating reasonable calibration. The NT-proBNP with clinical features model (Model 6) showed good calibration when applied to each validation data set, across the full range of the probability scale. The BNP with clinical features model (Model 5) was less well calibrated, though the 95% confidence intervals crossed the diagonal line in all cases except one (Cost ), when predicted risk of heart failure was between 0.2 and 0.4.
(Enlarge Image)
Figure 1.
Calibration plots for the clinical and natriuretic peptide model. The circles represent the observed proportion that had heart failure in the data set, and the vertical lines the 95% confidence interval around this proportion.
(Enlarge Image)
Figure 2.
The MICE rule.
Adaptation of the Clinical Decision Rule for Clinical Practice
The model was then split into a two-stage process: clinical features alone, and clinical features plus natriuretic peptide result. The Supplementary material, Figure S1 gives estimates of likelihood ratios and post-test probabilities for the combinations of clinical features alone, and shows that if the pre-test probability is set at 25%, the probability threshold of 20% is exceeded for all combinations except males and females without any of the features, and in females with ankle oedema. Increasing the pre-test probability to 35% made little difference to this result (though females with ankle oedema just crossed the threshold with a post-test probability of 21%).
This suggested a simple clinical rule abbreviated as MICE: Male, Infarction, Crepitations, Edema.
For the three groups recommended for further natriuretic peptide testing, test thresholds are shown in Table 3 at which the post-test probability of heart failure exceeds 20, 25, and 30%, representing different possible referral thresholds for echocardiography.
Performance of MICE
The performance of the one-stage model (clinical features plus natriuretic peptides) and the two-stage model (clinical features then natriuretic peptides as used in the MICE rule) in terms of sensitivity, specificity, and positive and negative predictive values is shown in Table 4. Sensitivity of the one-stage model using BNP varied in the different data sets between 67% and 86%, and using NT-proBNP between 77% and 92%. Sensitivity of the two-stage model varied between 81% and 89% for BNP and between 90% and 96% for NT-proBNP. Specificity of the one-stage model for BNP varied between 62% and 86%, and for NT-proBNP between 66% and 80%. Specificity of the two-stage model varied between 57% and 74% for BNP and 58% and 63% for NT-proBNP.
Source...