Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes.
Loading...
Embargo End Date
Authors
Cubuk, C
Garrett, A
Choi, S
King, L
Loveday, C
Torr, B
Burghel, GJ
Durkie, M
Callaway, A
Robinson, R
Drummond, J
Berry, I
Wallace, A
Eccles, D
Tischkowitz, M
Whiffin, N
Ware, JS
Hanson, H
Turnbull, C
CanVIG-Uk,
Garrett, A
Choi, S
King, L
Loveday, C
Torr, B
Burghel, GJ
Durkie, M
Callaway, A
Robinson, R
Drummond, J
Berry, I
Wallace, A
Eccles, D
Tischkowitz, M
Whiffin, N
Ware, JS
Hanson, H
Turnbull, C
CanVIG-Uk,
Document Type
Journal Article
Date
2021-07-06
Date Accepted
2021-06-17
Abstract
PURPOSE: Where multiple in silico tools are concordant, the American College of Medical Genetics and Genomics/Association for Molecular Pathology (ACMG/AMP) framework affords supporting evidence toward pathogenicity or benignity, equivalent to a likelihood ratio of ~2. However, limited availability of "clinical truth sets" and prior use in tool training limits their utility for evaluation of tool performance. METHODS: We created a truth set of 9,436 missense variants classified as deleterious or tolerated in clinically validated high-throughput functional assays for BRCA1, BRCA2, MSH2, PTEN, and TP53 to evaluate predictive performance for 44 recommended/commonly used in silico tools. RESULTS: Over two-thirds of the tool-threshold combinations examined had specificity of <50%, thus substantially overcalling deleteriousness. REVEL scores of 0.8-1.0 had a Positive Likelihood Ratio (PLR) of 6.74 (5.24-8.82) compared to scores <0.7 and scores of 0-0.4 had a Negative Likelihood Ratio (NLR) of 34.3 (31.5-37.3) compared to scores of >0.7. For Meta-SNP, the equivalent PLR = 42.9 (14.4-406) and NLR = 19.4 (15.6-24.9). CONCLUSION: Against these clinically validated "functional truth sets," there was wide variation in the predictive performance of commonly used in silico tools. Overall, REVEL and Meta-SNP had best balanced accuracy and might potentially be used at stronger evidence weighting than current ACMG/AMP prescription, in particular for predictions of benignity.
Citation
Genetics in Medicine
Source Title
Publisher
ELSEVIER SCIENCE INC
ISSN
1098-3600
eISSN
1530-0366