Enhancing Clinical Utility of AI Models in PCOS: Integrating Staged Modelling with uncertainty aware Triage and Conformal Prediction for Cost-Efficient Risk Stratification

Polycystic Ovary Syndrome (PCOS) is a common endocrine disorder affecting women of reproductive age, yet its diagnosis remains challenging due to reliance on costly investigations and limited diagnostic capacity in many clinical settings. We develop and assess a two-stage modelling pipeline in which Stage 1 uses low-cost demographic and routine clinical variables for initial screening, and Stage 2 augments these with laboratory and ultrasound features when escalation is warranted. Logistic Regression (LR) and Random Forest (RF) models are evaluated using calibration and classification metrics (AUC, accuracy, F1, precision, recall, Brier score, ECE), alongside clinical utility via decision curve analysis (DCA). To enhance safety and transparency, we integrate conformal prediction (CP) to provide finite-sample coverage guarantees and controlled abstention. Across train-test performance and out-of-fold evaluations, both models demonstrated consistent performance gains from Stage 1 to Stage 2. AUC increased by 6.9% for LR and increased by 7.4% for RF. LR exhibited more favourable calibration in most settings, while RF achieved higher precision, particularly after escalation. DCA showed higher net benefit for Stage 2 across clinically relevant thresholds. Feature sensitivity analysis indicated that a compact subset of inexpensive predictors preserved over 80% of maximal AUC, supporting cost-efficient screening. CP achieved 94.5% overall coverage with a 41.3% abstention rate, maintaining near-nominal validity across age and BMI subgroups. Capacity-constrained triage experiments showed that prioritising highest-risk cases maximised net benefit when resources were limited. These findings suggest that staged, uncertainty-informed modelling may offer practical steps toward clinically aligned and resource-aware AI support for PCOS assessment.