- Open Access
Predictive model for the probability of malignancy in solitary pulmonary nodules: a meta-analysis
Journal of Cardiothoracic Surgery volume 17, Article number: 102 (2022)
To date, multiple predictive models have been developed with the goal of reliably differentiating between solitary pulmonary nodules (SPNs) that are malignant and those that are benign. The present meta-analysis was conducted to assess the diagnostic utility of these predictive models in the context of SPN differential diagnosis.
The PubMed, Embase, Cochrane Library, CNKI, Wanfang, and VIP databases were searched for relevant studies published through August 31, 2021. Pooled data analyses were conducted using Stata v12.0.
In total, 20 retrospective studies that included 5171 SPNs (malignant/benign: 3662/1509) were incorporated into this meta-analysis. Respective pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic score values were 88% (95CI%: 0.84–0.91), 78% (95CI%: 0.74–0.80), 3.91 (95CI%: 3.42–4.46), 0.16 (95CI%: 0.12–0.21), and 3.21 (95CI%: 2.87–3.55), with an area under the summary receiver operating characteristic curve value of 86% (95CI%: 0.83–0.89). Significant heterogeneity among studies was detected with respect to sensitivity (I2 = 89.07%), NLR (I2 = 87.29%), and diagnostic score (I2 = 72.28%). In a meta-regression analysis, sensitivity was found to be impacted by the standard reference in a given study (surgery and biopsy vs. surgery only, P = 0.02), while specificity was impacted by whether studies were blinded (yes vs. unclear, P = 0.01). Sensitivity values were higher when surgery and biopsy samples were used as a standard reference, while unclear blinding status was associated with increased specificity. No significant evidence of publication bias was detected for the present meta-analysis (P = 0.539).
The results of this meta-analysis demonstrate that predictive models can offer significant diagnostic utility when establishing whether SPNs are malignant or benign.
As thoracic computed tomography (CT) scans are routinely conducted during physical examinations for certain patient populations, solitary pulmonary nodules (SPNs) of uncertain clinical significance are relatively common clinical entities. These SPNs can be malignant or correspond to early-stage lung cancer , with an estimated 55–77% being malignant and with rising odds of malignancy with increasing SPN diameter [2,3,4].
While pathological diagnosis is generally the definitive approach to SPN differentiation, it necessitates invasive biopsy or surgical resection procedures. In order to avoid unnecessary invasive diagnostic procedures when possible, comprehensive alternative approaches to SPN evaluation are required [5,6,7]. Differential SPN diagnosis cannot be achieved successfully through the assessment of a single radiological or clinical feature, underscoring the need for the development of predictive models capable of gauging the likelihood that a given SPN is malignant. The first such predictive model was reported in 1999 by the Mayo clinic , with many more such models having been developed to date by multiple international research teams [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30].
While promising, these predictive models exhibit significant variability among studies with respect to reported sensitivity and specificity values. These differences may be attributable to sample sizes and to whether models incorporated tumor marker tests or positron emission tomography (PET)/CT imaging results. There is thus a clear need for larger-scale analyses aimed at assessing the overall diagnostic value of these models.
As such, we herein conducted a meta-analysis designed to assess the diagnostic utility of predictive models used for the differential diagnosis of SPNs.
Materials and methods
The PubMed, Embase, Cochrane Library, CNKI, Wanfang, and VIP databases were searched for relevant studies published as of August 31, 2021 using the following search strategy: ((((((diagnosis[Title/Abstract]) OR (analysis[Title/Abstract])) OR (probability[Title/Abstract])) OR (differential[Title/Abstract])) OR (predictive[Title/Abstract])) AND (model[Title/Abstract])) AND (((pulmonary nodule[Title/Abstract]) OR (lung nodule[Title/Abstract])) OR (SPN[Title/Abstract])). This meta-analysis was registered at https://inplasy.com/ (No. INPLASY2021100006).
Studies eligible for inclusion were: (1) studies assessing the differential diagnosis of benign and malignant SPNs; (2) studies of SPNs ≤ 30 mm in size; (3) studies in which predictive models were developed and provided; (4) studies in which sensitivity and specificity were provided. Studies were excluded if they were reviews, case reports, or non-human studies.
Data extraction and quality assessment
Relevant data were independently extracted from included studies by two researchers, with any disagreements being resolved by a third researcher. Extracted data included authors, publication year, publication country, study design, blinding status, sample size, SPN size, reference standards, predictive model contents, and true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN) results. Risk of bias was evaluated with the quality assessment of diagnostic accuracy studies (QUADAS-2) tool .
SPNs were defined as isolated round lung lesions ≤ 3 cm in size not associated with atelectasis, pleural effusion, or mediastinal lymphadenopathy . A TP result was defined as one in which both the predictive model and final diagnosis were indicative of malignancy, while an FP result was defined as one in which the predictive model indicated that a lesion was malignant whereas the final diagnosis indicated that it was benign. A TN result was defined as one in which both the predictive model and final diagnosis were indicative of a benign SPN, while an FN result was defined as one in which a predictive model indicated that an SPN was benign but the final diagnosis for that lesion indicated it was malignant.
The sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic score, and summary receiver operating characteristic (SROC) curve were pooled using Stata v12.0 (Stata Corporation, TX, USA).
A PLR > 5 or an NLR < 0.2 were considered to be indicative of high diagnostic ability for a given predictive model. Diagnostic ability was also considered to be good if the area under the SROC curve (AUC) was > 80% .
Heterogeneity was assessed with the I2 index, with an I2 > 50% being indicative of significant heterogeneity. A meta-regression was used to detect potential sources of heterogeneity, and subgroup analyses were conducted based upon these identified sources. Deeks’ funnel plots were used to gauge potential publication bias, and P < 0.05 was the threshold of significance for this study.
The study selection process for the present study is outlined in Fig. 1. Ultimately, 20 studies were included in the final analysis, all of which were retrospective in design and conducted by Chinese research teams. We have found 4 eligible studies which were from out of China [7,8,9,10]. However, these studies had insufficient data to construct a 2 × 2 table, and therefore, these studies were excluded from this meta-analysis. These 20 studies (Table 1) included 5171 total SPNs (malignant: 3662; benign: 1509). PET/CT results were included in 7 of these studies [14, 15, 18,19,20, 22, 24], while 6 included tumor marker results [11, 16, 23, 24, 27, 29]. Moreover, 10 studies had predictive models consisting of > 4 factors [14,15,16, 20, 21, 23,24,25,26,27, 30]. The details of each predictive model are outlined in Table 2, and raw TP, FP, TN, and FN data are compiled in Table 3.
The QUADAS-2 tool was used to assess potential bias in the present meta-analysis (Fig. 2). Of the 20 included studies, 11 failed to indicate whether patients were enrolled in a consecutive manner [11, 13, 15, 18–20, 22, 24. 25, 28, 30], while 14 provided unclear information pertaining to the blinding status [11,12,13,14,15,16, 19, 20, 23,24,25,26,27,28, 30]. All studies described the reference standard used to confirm the diagnosis.
Pooled sensitivity, specificity, PLR, NLR, and diagnostic score values were 88% (95CI%: 0.84–0.91, Table 4), 78% (95CI%: 0.74–0.80, Table 4), 3.91 (95CI%: 3.42–4.46, Table 4), 0.16 (95CI%: 0.12–0.21, Table 4), and 3.21 (95CI%: 2.87–3.55, Table 4), respectively. Significant heterogeneity was detected with respect to sensitivity (I2 = 89.07%), NLR (I2 = 87.29%), and diagnostic score (I2 = 72.28%). The AUC value was 86% (95CI%: 0.83–0.89, Fig. 3). The SROC curve consistent with substantial deviation from a shoulder-like appearance, exhibiting the potential absence of any threshold effect.
The results of a meta-regression analysis are shown in Table 5. Sensitivity was impacted by the reference standards utilized in a given study (surgery only vs. surgery and biopsy, P = 0.02). Specificity was impacted by whether blinding was employed (yes vs. unclear, P = 0.01). Sample size, the number of predictive factors, whether models incorporated PET/CT results, and whether models incorporated tumor marker results had no impact on the final diagnostic results.
Subgroup analyses were conducted based upon differences in reference standard and blinding situations (Table 4). Higher sensitivity was observed in the subgroup in which surgery and biopsy were used for reference sample collection, while higher specificity was evident in the subgroup in which the blinding situation was unclear.
Deeks’ funnel plot asymmetry test did not reveal any evidence of publication bias in the present meta-analysis (P = 0.539).
We found that lobulation sign was the most common feature of the predictive models and it occurred in 11 of the 20 studies [11, 13,14,15,16, 18, 22,23,24, 26, 30]. The TP, FP, TN, and FN data of lobulation sign could be extracted from 7 studies [11, 16, 22,23,24, 26, 30]. Pooled sensitivity, specificity, PLR, NLR, and diagnostic score values were 57% (95CI%: 0.38–0.74), 80% (95CI%: 0.62–0.91), 2.84 (95CI%: 1.76–4.63), 0.54 (95CI%: 0.40–0.72), and 1.66 (95CI%: 1.17–2.16), respectively. The AUC value was 74% (95CI%: 0.70–0.78).
In this meta-analysis, we explored the diagnostic utility of predictive models in the context of SPN differential diagnosis. Overall, we found these models to exhibit robust predictive value with a high AUC value of 86%. As the NLR value (0.16) was less than 0.2, this indicated that lower predictive scores were associated with the satisfactory prediction of benign SPNs. However, the pooled PLR value (3.91) was less than 5, indicating that higher predictive scores were only moderately predictive of malignant SPNs.
In CT-based analyses, spiculation and calcification signs are commonly leveraged as predictive factors when evaluating SPNs, and are routinely incorporated into developed predictive models [32, 33]. Spiculation sign is generally indicative of malignant SPNs, whereas calcified nodules are more likely to be benign. However, one prior meta-analysis found spiculation sign to only exhibit moderate diagnostic accuracy when used to evaluate SPNs (AUC = 76%) . In a separate meta-analysis, calcification was found to be a good predictor of benign SPN status (PLR = 6.06) , although the overall diagnostic utility of such calcification was somewhat limited (AUC = 65%) . These results suggest that any individual sign only offers limited diagnostic value in the evaluation of SPNs. Combining these signs together, however, may significantly improve the overall diagnostic value of developed models.
Sensitivity values were impacted by the reference standard used in included studies (P = 0.02), with higher sensitivity values being reported when the references standard included surgery and biopsy samples. While benign are SPNs are generally confirmed via surgical resection, malignancy SPNs can be confirmed via both surgery and biopsy . When researchers only focused on surgically-confirmed SPNs in their studies, this markedly reduced the malignant SPN sample size, thereby constraining the sensitivity of the resultant models.
Specificity values were found to be impacted by the blinding situation for included studies (P = 0.01), with an unclear blinding situation being associated with higher specificity. This may be attributable to the relatively high number of studies with unclear blinding (n = 15) as compared to the number of studies with definitive blinding (n = 5).
In one prior meta-analysis, tests for carcinoembryonic antigen (CEA) alone were found to be associated with moderate diagnostic utility (AUC = 77%) when differentiating between malignant and benign SPNs . However, the incorporation of other tumor marker tests can improve the overall accuracy of developed diagnostic models for SPN evaluation . PET/CT also exhibits high diagnostic ability when used to assess SPNs . Even so, in the present meta-analysis, the incorporation of tumor marker and PET/CT tests did not increase the overall diagnostic utility of developed predictive models. This may be attributable to the fact that there were relatively few studies that included tumor marker (n = 6) and PET/CT (n = 7) tests in the overall analysis.
In this meta-analysis, lobulation sign was the most common feature of the included predictive models. However, the area under the SROC curve was only 74%, which was less than that (86%) made by the predictive models. This finding indicated that predictive model could provide more comprehensive analysis for SPNs than a single feature did.
There are certain limitations to this meta-analysis. For one, the major limitation is the fact that all included studies were retrospective nature, and this caused the major bias in the results of this meta-analysis. Additional prospective studies will thus be critical to validate and expand these results. Secondly, many of these studies failed to clarify whether consecutive patients were enrolled, potentially influencing the diagnostic accuracy of the developed predictive models. Third, none of these studies employed a CT-based follow-up approach to confirm the identity of SPNs that were diagnosed as benign. While surgical resection can provide the most precise diagnostic information pertaining to benign lesions, CT follow-up can also be accepted for final diagnosis . The absence of CT-based follow-up may have thus impacted the reported diagnostic accuracy. Fourth, none of these models included magnetic resonance imaging (MRI)-based results. While MRI scans are not commonly used to evaluate lung disease, some prior studies have suggested that they may offer value as a means of distinguishing between SPNs that are malignant and those that are benign . Lastly, all included studies were from China and this may further increase the risk of bias. Although China is the country with the world largest population, additional studies from other countries are still needed.
In summary, this meta-analysis demonstrated that predictive models offer substantial diagnostic value when establishing whether SPNs are malignant or benign, although further research will be required to confirm these findings.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Positive likelihood ratio
Negative likelihood ratio
Summary receiver operating characteristic
Magnetic resonance imaging
Solitary pulmonary nodules
Cruickshank A, Stieler G, Ameer F. Evaluation of the solitary pulmonary nodule. Intern Med J. 2019;49:306–15.
Choo JY, Park CM, Lee NK, et al. Percutaneous transthoracic needle biopsy of small (≤ 1 cm) lung nodules under C-arm cone-beam CT virtual navigation guidance. Eur Radiol. 2013;23:712–9.
Weir-McCall JR, Joyce S, Clegg A, et al. Dynamic contrast-enhanced computed tomography for the diagnosis of solitary pulmonary nodules: a systematic review and meta-analysis. Eur Radiol. 2020;30:3310–23.
Chan EY, Gaur P, Ge Y, et al. Management of the solitary pulmonary nodule. Arch Pathol Lab Med. 2017;141:927–31.
Truong MT, Ko JP, Rossi SE, et al. Update in the evaluation of the solitary pulmonary nodule. Radiographics. 2014;34:1658–79.
Majid A, Fernandez-Bussy S, Folch E. Interventional pulmonology and solitary pulmonary nodule. Arch Bronconeumol. 2018;54:497–8.
Talwar A, Rahman NM, Kadir T, et al. A retrospective validation study of three models to estimate the probability of malignancy in patients with small pulmonary nodules from a tertiary oncology follow-up centre. Clin Radiol. 2017;72:177.e1-177.e8.
Swensen SJ, Silverstein MD, Edell ES, et al. Solitary pulmonary nodules: clinical prediction model versus physicians. Mayo Clin Proc. 1999;74:319–29.
Herder GJ, van Tinteren H, Golding RP, et al. Clinical prediction model to characterize pulmonary nodules: validation and added value of 18F-fluorodeoxyglucose positron emission tomography. Chest. 2005;128:2490–6.
Yonemori K, Tateishi U, Uno H, et al. Development and validation of diagnostic prediction model for solitary pulmonary nodules. Respirology. 2007;12:856–62.
Cao L, Tao YJ, Zhao YL, et al. Analysis in risk factors of solitary pulmonary nodules. J Clin Med Pract. 2021;25:38–40.
Chen W, Zhu D, Chen H, et al. Predictive model for the diagnosis of benign/malignant small pulmonary nodules. Medicine (Baltimore). 2020;99: e19452.
Chen J, Ye XD, Ye JD, et al. Establishment and verification of a mathematical model for predicting malignancy of solitary pulmonary nodules. 2016;22:573–7.
Chen YM, Tong LJ, Shi YP, et al. Clinical verification of a mathematical model for diagnosing SPN with 18 F-DG PET/CT. J Med Imaging. 2013;23:1014–7.
Cheng Y, Wang ZG, Yang GJ, et al. A model of malignant risk prediction for solitary pulmonary nodules on 18F-FDG PET/CT: building and estimating. Chin J Nucl Med Mol Imaging. 2019;39:129–32.
Dong J, Sun N, Li J, et al. Development and validation of clinical diagnostic models for the probability of malignancy in solitary pulmonary nodules. Thorac Cancer. 2014;5:162–8.
Hu H, Wang Q, Tang H, et al. Multi-slice computed tomography characteristics of solitary pulmonary ground-glass nodules: differences between malignant and benign. Thorac Cancer. 2016;7:80–7.
Lin J, Tang K, Yin WW, et al. The comparative study of 18F-FDG PET/CT and mathematical prediction model in diagnosis of solitary pulmonary nodule. J Wenzhou Med Univ. 2015;45:354–8.
Ma N, Zhao M, Tian RR, et al. Comparative analysis of different diagnostic methods based on PET/CT in qualitative diagnosis of solitary pulmonary nodules. Chin J Interv Imaging. 2020;17:233–7.
Tian R, Su MG, Tian Y, et al. Development of a predicting model to estimate the probability of malignancy of solitary pulmonary nodules. Sichuan Da Xue Xue Bao Yi Xue Ban. 2012;43:404–8.
Wang X, Xu YH, Du ZY, et al. Risk factor analysis of the patients with solitary pulmonary nodules and establishment of a prediction model for the probability of malignancy. Zhonghua Zhong Liu Za Zhi. 2018;40:115–20.
Xiang Y, Sun Y, Gao W, et al. Establishment of a predicting model to evaluate the probability of malignancy or benign in patients with solid solitary pulmonary nodules. Zhonghua Yi Xue Za Zhi. 2016;96:1354–8.
Xiao F, Yu Q, Zhang Z, et al. Establishment and verification of a novel predictive model of malignancy for non-solid pulmonary nodules. Zhongguo Fei Ai Za Zhi. 2019;22:26–33.
Xu L, Zhang LB, Chen G. The tumor markers in diagnosis of solitary pulmonary nodule. J Clin Intern Med. 2020;37:368–70.
Yang DS, Li Y, Jiang GC, et al. Clinical vedfication and appHcation of mathem atical models for predicting the probability of malignant or benign in patients with solitary pulmonary nodule. Chin J Thorac Cardiovasc Surg. 2012;28:82–5.
Yu W, Ye B, Xu L, et al. Establishment of a clinical prediction model of solid solitary pulmonary nodules. Zhongguo Fei Ai Za Zhi. 2016;19:705–10.
Zhang M, Zhuo N, Guo Z, et al. Establishment of a mathematic model for predicting malignancy in solitary pulmonary nodules. J Thorac Dis. 2015;7:1833–41.
Zhang XH, Chen C, Zeng H, et al. Establishment of clinical prediction model to estimate the probability of malignancy in patients with solitary pulmonary nodules. Pract J Cancer. 2016;31:59–62.
Zhao HC, Xu QS, Shi YB, et al. Clinical-radiological predictive model in differential diagnosis of small (≤ 20 mm) solitary pulmonary nodules. BMC Pulm Med. 2021;21:281.
Zhong CH, Shi HC, Shu YS, et al. Establishment and clinical verification of a mathematical model for predicting the probability of malignancy or begin in patients with solitary pulmonary nodules. J Clin Med Pract. 2017;21:82–93.
Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36.
Zhou C, Liu XB, Gan XJ, et al. Calcification sign for prediction of benignity in pulmonary nodules: a meta-analysis. Clin Respir J. 2021. https://doi.org/10.1111/crj.13410.
Li Y, Wang T, Fu YF, et al. Computed tomography-based spiculated sign for prediction of malignancy in lung nodules: a meta-analysis. Clin Respir J. 2020;14:1113–21.
Huang YY, Cheng H, Li GC. Computed tomography-guided core needle biopsy for lung nodules: low-dose versus standard-dose protocols. Wideochir Inne Tech Maloinwazyjne. 2021;16:355–61.
Li L, Guo C, Wan JL, et al. The use of carcinoembryonic antigen levels to predict lung nodule malignancy: a meta-analysis. Acta Clin Belg. 2020;23:1–6. https://doi.org/10.1080/17843286.2020.1797330.
Mosmann MP, Borba MA, de Macedo FP, et al. Solitary pulmonary nodule and (18)F-FDG PET/CT. Part 2: accuracy, cost-effectiveness, and current recommendations. Radiol Bras. 2016;49:104–11.
Fu YF, Jiang LH, Wang T, et al. Nonspecific benign pathological results on computed tomography-guided lung biopsy: A predictive model of true negatives. J Cancer Res Ther. 2019;15:1464–70.
Li B, Li Q, Chen C, et al. A systematic review and meta-analysis of the accuracy of diffusion-weighted MRI in the detection of malignant pulmonary nodules and masses. Acad Radiol. 2014;21:21–9.
Ethics approval and consent to participate
This is a meta-analysis and ethics approval and consent to participate are not required.
Consent for publication
This is a meta-analysis and consent for publication is not required.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Chen, G., Bai, T., Wen, LJ. et al. Predictive model for the probability of malignancy in solitary pulmonary nodules: a meta-analysis. J Cardiothorac Surg 17, 102 (2022). https://doi.org/10.1186/s13019-022-01859-x
- Predictive model
- Solitary pulmonary nodule