Quantized multi-task learning for context-specific representations of gene network dynamics
Chen, H.; Venkatesh, M. S.; Gomez Ortega, J.; Mahesh, S. V.; Nandi, T. N.; Madduri, R. K.; Pelka, K.; Theodoris, C. V.
Show abstract
While often represented as static entities, gene networks are highly context-dependent. Here, we developed a multi-task learning strategy to yield context-specific representations of gene network dynamics. We assembled a corpus comprising [~]103 million human single-cell transcriptomes from a broad range of tissues and diseases and performed a two stage pretraining, first with non-malignant cells to generate a foundational model and then with continual learning on cancer cells to tune the model to the cancer domain. We performed multi-task learning with the foundational model to learn context-specific representations of a broad range of cell types, tissues, developmental stages, and diseases. We then leveraged the cancer-tuned model to jointly learn cell states and predict tumor-restricting factors within the colorectal tumor microenvironment. Model quantization allowed resource-efficient fine-tuning and inference while preserving biological knowledge. Overall, multi-task learning enables context-specific disease modeling that can yield contextual predictions of candidate therapeutic targets for human disease.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- GraphVelo allows for accurate inference of multimodal velocities and molecular mechanisms for single cells 98%
- High-throughput single-cell chromatin accessibility CRISPR screens enable unbiased identification of regulatory networks in cancer 98%
- Spatiotemporal Regulatory Logics of Mouse Gastrulation 97%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.