Assessing the Performance of LLMs in Multimodal Information Extraction for Biological Research: A Case Study on LLPS
Chin, K. Y.; Fujii, S.; Ishida, S.; Terayama, K.
Show abstract
Advances in experimental techniques have expanded the volume of biological data. This has increased the demand for structured information extraction from papers, with large language models (LLMs) considered promising. However, challenges remain, including limited validation in biology and unclear applicability to multimodal tasks that integrate text with domain-specific figures, such as microscopic images and scatter plots. Here, we developed a multimodal LLM (MLLM)-based workflow to extract the experimental conditions and phase status from the text and figures of experimental papers on liquid-liquid phase separation and validated the effect of various inputs, prompts, and MLLM types. As a result, the Gemini 2.5 Pro-based extraction achieved an F1-score of 0.847 by processing each figure as a processing unit and inputting domain-specific prompts reflecting manual extraction guidance. This study demonstrates the potential and limitations of MLLMs for extracting biological information and provides insights for advancing multimodal approaches in biology.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- pDeep3: Towards More Accurate Spectrum Prediction with Fast Few-Shot Learning 91%
- Continuous, Low Latency Estimation of the Size and Shape of Single Proteins from Real-Time Nanopore Data 91%
- Well-Paired-Seq2: High-Throughput and High-Sensitivity Strategy for Characterizing Low RNA-Content Cell/Nucleus Transcriptomes 90%
Similar papers in this journal
Similar papers in this journal
- DNAi: an open-source AI tool for unbiased DNA fiber analysis 93%
- MarcoPolo: a clustering-free approach to the exploration of differentially expressed genes along with group information in single-cell RNA-seq data 92%
- OmicsFootPrint: a framework to integrate and interpret multi-omics data using circular images and deep neural networks 92%
Similar papers in this journal
- Gra-CRC-miRTar: The pre-trained nucleotide-to-graph neural networks to identify potential miRNA targets in colorectal cancer 92%
- DeepNeuropePred: a robust and universal tool to predict cleavage sites from neuropeptide precursors by protein language model 91%
- Extracting physical characteristics of higher-order chromatin structures from 3D image data 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.