Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur Accéder directement au contenu
Article Dans Une Revue Systematic Reviews Année : 2019

Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy

Résumé

Background: The large and increasing number of new studies published each year is making literature identification in systematic reviews ever more time-consuming and costly. Technological assistance has been suggested as an alternative to the conventional, manual study identification to mitigate the cost, but previous literature has mainly evaluated methods in terms of recall (search sensitivity) and workload reduction. There is a need to also evaluate whether screening prioritization methods leads to the same results and conclusions as exhaustive manual screening. In this study, we examined the impact of one screening prioritization method based on active learning on sensitivity and specificity estimates in systematic reviews of diagnostic test accuracy. Methods: We simulated the screening process in 48 Cochrane reviews of diagnostic test accuracy and rerun 400 meta-analyses based on a least 3 studies. We compared screening prioritization (with technological assistance) and screening in randomized order (standard practice without technology assistance). We examined if the screening could have been stopped before identifying all relevant studies while still producing reliable summary estimates. For all meta-analyses, we also examined the relationship between the number of relevant studies and the reliability of the final estimates. Results: The main meta-analysis in each systematic review could have been performed after screening an average of 30% of the candidate articles (range 0.07 to 100%). No systematic review would have required screening more than 2308 studies, whereas manual screening would have required screening up to 43,363 studies. Despite an average 70% recall, the estimation error would have been 1.3% on average, compared to an average 2% estimation error expected when replicating summary estimate calculations. Conclusion: Screening prioritization coupled with stopping criteria in diagnostic test accuracy reviews can reliably detect when the screening process has identified a sufficient number of studies to perform the main meta-analysis with an accuracy within pre-specified tolerance limits. However, many of the systematic reviews did not identify a sufficient number of studies that the meta-analyses were accurate within a 2% limit even with exhaustive manual screening, i.e., using current practice .
Fichier principal
Vignette du fichier
Measuring-the-impact-of-screeningautomation-on-meta-analyses-of-diagnostictest-accuracy.pdf (1.08 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-02471032 , version 1 (07-02-2020)

Identifiants

Citer

Christopher R Norman, Mariska M G Leeflang, Raphaël Porcher, Aurelie Neveol. Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy. Systematic Reviews, 2019, ⟨10.1186/s13643-019-1162-x⟩. ⟨hal-02471032⟩
43 Consultations
145 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More