Mind the gap: preventing circularity in missense variant prediction
Heijl, S.; Vroling, B.; van den Bergh, T.; Joosten, H.-J.
Show abstract
Despite advances in the field of missense variant effect prediction, the real clinical utility of current computational approaches remains rather limited. There is a large difference in performance metrics reported by developers and those observed in the real world. Most currently available predictors suffer from one or more types of circularity in their training and evaluation strategies that lead to overestimation of predictive performance. We present a generic strategy that is independent of dataset properties and algorithms used, to deal with circularity in the training phase. This results in more robust predictors and evaluation scores that accurately reflect the real-world performance of predictive models. Additionally, we show that commonly used training methods can have an adverse impact on model performance and lead to gross overestimation of true predictive performance.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Selecting the most important self-assessed features for predicting conversion to Mild Cognitive Impairment with Random Forest and Permutation-based methods 94%
- PIPENN-EMB: ensemble net and protein embeddings generalise protein interface prediction beyond homology 94%
- Synthetic data for privacy-preserving clinical risk prediction 94%
Similar papers in this journal
- MENDELSEEK: An algorithm that predicts Mendelian Genes and elucidates what makes them special 94%
- Paying Attention to Attention: High Attention Sites as Indicators of Protein Family and Function in Language Models 94%
- Designing diverse and high-performance proteins with a large language model in the loop 93%
Similar papers in this journal
- Soft Windowing Application to Improve Analysis of High-throughput Phenotyping Data 95%
- 3Cnet: Pathogenicity prediction of human variants using knowledge transfer with deep recurrent neural networks 95%
- Multi-Omic Graph Diagnosis (MOGDx) : A data integration tool to perform classification tasks for heterogeneous diseases 94%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.