Comparative Assessment of Large Language Models for Microbial Phenotype Annotation
Muench, P. C.; Safaei, N.; Han, Y.; Binder, M.; Robertson, G.; Mreches, R.; Franzosa, E. A.; Huttenhower, C.; McHardy, A. C.
Show abstract
Large language models (LLMs) are increasingly used to extract knowledge from text, yet their coverage and reliability in biology remain unclear. Microbial phenotypes are especially important to assess, as comprehensive data remain sparse except for well-studied organisms and they underpin our understanding of microbial characteristics, functional roles, and applications. Here, we systematically assessed the biological knowledge encoded in publicly available LLMs for structured phenotype annotation of microbial species. We evaluated the performance of over 50 LLMs, including state-of-the-art models such as Claude Sonnet 4 and the GPT-5 family of models. Across phenotypes, LLMs reached accurate assignments for many species, but performance varied widely by model and trait, and no single model dominated. Model self-reported confidence is informative, with higher confidence aligning with higher accuracy, and can be used to prioritize phenotype assignment, effectively distinguishing between high-and low-confidence inferences. Overall, our study outlines the utility and limitations of text-based LLMs for phenotype characterization in microbiology.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- APOLLO: A genome-scale metabolic reconstruction resource of 247,092 diverse human microbes spanning multiple continents, age groups, and body sites 95%
- Guiding the Refinement of Biochemical Knowledgebases with Ensembles of Metabolic Networks and Machine Learning 94%
- Metabolic rules of microbial community assembly 93%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.