Predicting IVF live birth probabilities using machine learning, center-specific and national registry-based models

Ongoing improvement of pretreatment live birth prognostication for in vitro fertilization (IVF) is critical for informing fertility patients treatment decisions, advocating for IVF coverage and supporting value-based IVF care. The US national registry Society for Assisted Reproductive Technology (SART) IVF live birth prediction (LBP) model (SART model) has been widely adopted for its prognostic support without external validation or utilization studies. We conducted a retrospective model validation study to compare the IVF LBP performance of machine learning, center-specific (MLCS) models versus the SART model in 6 unrelated US fertility centers using their respective center-specific test sets comprising an aggregate of 4,635 patients first-IVF cycle data. Compared to the SART model, MLCS2 showed higher median Precision Recall AUC at 0.75 (IQR 0.73, 0.77) vs. 0.69 (IQR 0.68, 0.71), p<0.05 and higher median F1 Score across LBP thresholds. Further, MLCS1 showed no evidence of data drift when validated using out-of-time test data from a later period. Reclassification analysis showed that MLCS2 models assigned more appropriate and higher IVF LBPs compared to the SART model, which underestimated patient prognoses (continuous net reclassification index: 18.3%, p<0.0001). Overall, MLCS2 and SART models assigned 30% of patients to differential prognostic groups, with MLCS2 assigning 26% of patients to a higher LBP category compared to the SART model. Importantly, MLCS2 models identified 11% of patients to have LBP [&ge;] 75%, whereas the SART model detected none. This group had a live birth rate of 81%. We recommend testing a larger sample of fertility centers to further evaluate MLCS model benefits and limitations.