Verification Debt in Automated Biological Research Workflows: A Proof-of-Concept
Gopalan, S.; Narayanan, S.
Show abstract
Recent biological research has shown significant interest and progress towards adoption of agentic AI systems. Domain specific agentic platforms with access to large collections of biological tools with code generation and/or querying capabilities, such as Biomni (illustrative case in point), are representative of the capability to orchestrate automated research workflows. Such platforms enable collaborative AI workspace for drug discovery, genomics, bioinformatics, literature research, and experimental planning for biologists and researchers. When such platforms are used for the aforementioned workflows, individually validating every agentic decision and the research process by which it was reached becomes challenging due to the scale of decisions involved and oftentimes, insufficient disclosure of assumptions and methodological choices, limited traceability of evidence, and interpretive judgments embedded throughout the workflow, giving rise to verification debt. Verification debt in agentic environments could result in unexpected failures and consequences in downstream experiments. This study focuses on empirical qualitative characterization of verification debt in agentic (prompt to outcomes) workflows. Biomni was chosen the testing AI agentic platform in this study mainly due to the descriptive traces it provides to track every agentic step. Verification debt is examined through two complementary angles: the individual agentic decision steps that carry claims and evidence, and the underlying research process by which the agent arrives at and reports them. This study attempts to characterize verification debt with 10 test cases (covering proteomics, genomics and drug discovery) with varying degree of biological nuance across three identical and independent runs. Verification debt is represented through 9 decision-step metrics and 13 research-process metrics, and also propose the VERIFY framework for evaluating scientific verifiability and validity in agentic workflows.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Extraction of biological terms using large language models enhances the usability of metadata in the BioSample database 92%
- FAIR Data Station for Lightweight Metadata Management & Validation of Omics Studies 92%
- A workflow reproducibility scale for automatic validation of biological interpretation results. 92%
Similar papers in this journal
- A Hybrid Modeling Framework for Predictive Digital Twins of CHO Cell Culture 90%
- Topological embedding and directional feature importance in ensemble classifiers for multi-class classification 89%
- Benchmarking feature selection and feature extraction methods to improve the performances of machine-learning algorithms for patient classification using metabolomics biomedical data. 89%
Similar papers in this journal
- The Mastery Rubric for Bioinformatics: supporting design and evaluation of career-spanning education and training 92%
- Knowledge Beacons: Web Service Workflow for FAIR Data Harvesting of Distributed Biomedical Knowledge 92%
- Understanding signaling and metabolic paths using semantified and harmonized information about biological interactions 91%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.