Systematic Evaluation of AlphaFold2 and OpenFold3 on Protein-Peptide Complexes

Protein-peptide interactions are important mediators of diverse biological processes. While deep learning has revolutionized protein structure prediction, comparative evaluation of these methods, specifically for protein-peptide complexes, remains an area of active investigation. Here, we present a systematic benchmarking of AlphaFold2 (AF2) and OpenFold3 (OF3) on a curated, non-redundant dataset of 271 protein-peptide complexes evaluated under CAPRI peptide criteria, partitioned into disordered (IDR) and structured (Non-IDR) peptide subsets. Results show that AF2 consistently outperformed OF3 across both subsets in overall success rate and proportion of high-quality models, while both methods exhibited comparable global fold prediction accuracy. We further demonstrate that AF2 exhibited memorization on a large set of protein-peptide complexes that were in its training data. Analysis of built-in and post-hoc confidence scores demonstrated that PAE-derived metrics, particularly pDockQ2, LIS, and ipSAE, provided the most reliable proxies for structural accuracy in AF2 predictions, whereas OF3s PAE distributions substantially diminished the discriminative power of its derived scores. Furthermore, we find that canonical DockQ threshold cutoffs for protein-protein complexes are not directly transferable to protein-peptide complexes, underscoring the need for method- and dataset-specific calibration. Peptide sequence composition and length were identified as potential modulators of prediction success, with glycine-rich short peptides and long receptors posing challenges to both methods. Collectively, these findings establish a peptide-specific evaluation framework and highlight the need for dataset/method-calibrated metrics to support the continued development of structure prediction tools for protein-peptide interactions.