Cross-Modal Autoencoder Framework Learns Holistic Representations of Cardiovascular State
Radhakrishnan, A.; Friedman, S. F.; Khurshid, S.; Ng, K.; Batra, P.; Lubitz, S. A.; Philippakis, A. A.; Uhler, C.
Show abstract
A fundamental challenge in diagnostics is integrating multiple modalities to develop a joint characterization of physiological state. Using the heart as a model system, we develop a cross-modal autoencoder framework for integrating distinct data modalities and constructing a holistic representation of cardio-vascular state. In particular, we use our framework to construct such cross-modal representations from cardiac magnetic resonance images (MRIs), containing structural information, and electrocardiograms (ECGs), containing myoelectric information. We leverage the learned cross-modal representation to (1) improve phenotype prediction from a single, accessible phenotype such as ECGs; (2) enable imputation of hard-to-acquire cardiac MRIs from easy-to-acquire ECGs; and (3) develop a framework for performing genome-wide association studies in an unsupervised manner. Our results provide a framework for integrating distinct diagnostic modalities into a common representation that better characterizes physiologic state.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Inferring spatial single-cell-level interactions through interpreting cell state and niche correlations learned by self-supervised graph transformer 93%
- Construction of a 3D whole organism spatial atlas by joint modeling of multiple slices 93%
- Hypergraph factorisation for multi-tissue gene expression imputation 93%
Similar papers in this journal
- Bi-level Graph Learning Unveils Prognosis-Relevant Tumor Microenvironment Patterns in Breast Multiplexed Digital Pathology 93%
- Single-Cell Multi-Modal GAN (scMMGAN) reveals spatial patterns in single-cell data from triple negative breast cancer 93%
- Generating hard-to-obtain information from easy-to-obtain information: applications in drug discovery and clinical inference 93%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.