Meta-analysis of (single-cell method) benchmarks reveals the need for extensibility and interoperability
Sonrel, A.; Luetge, A.; Soneson, C.; Mallona Gonzalez, I.; Germain, P.-L.; Knyazev, S.; Gilis, J.; Gerber, R.; Seurinck, R.; Paul, D.; Sonder, E.; Crowell, H. L.; Fanaswala, I.; Al Ajami, A.; Heidari, E.; Schmeing, S.; Milosavljevic, S.; Saeys, Y.; Mangul, S.; Robinson, M. D.
Show abstract
Computational methods represent the lifeblood of modern molecular biology. Benchmarking is important for all methods, but with a focus here on computational methods, benchmarking is critical to dissect important steps of analysis pipelines, formally assess performance across common situations as well as edge cases, and ultimately guide users on what tools to use. Benchmarking can also be important for community building and advancing methods in a principled way. We conducted a meta-analysis of recent single-cell benchmarks to summarize the scope, extensibility, neutrality, as well as technical features and whether best practices in open data and reproducible research were followed. The results highlight that while benchmarks often make code available and are in principle reproducible, they remain difficult to extend, for example, as new methods and new ways to assess methods emerge. In addition, embracing containerization and workflow systems would enhance reusability of intermediate benchmarking results, thus also driving wider adoption.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.