MOL-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective
Yang, J.; Zheng, K.; Long, S.; Nie, Z.; Zhang, M.; Dai, X.; Ma, W.-Y.; Zhou, H.
Show abstract
3D molecular representation learning has gained tremendous interest and achieved promising performance in various downstream tasks. A series of recent approaches follow a prevalent framework: an encoder-only model coupled with a coordinate denoising objective. However, through a series of analytical experiments, we prove that the encoderonly model with coordinate denoising objective exhibits inconsistency between pre-training and downstream objectives, as well as issues with disrupted atomic identifiers. To address these two issues, we propose MO_SCPLOWOLC_SCPLOW-AE for molecular representation learning, an auto-encoder model using positional encoding as atomic identifiers. We also propose a new training objective named 3D Cloze Test to make the model learn better atom spatial relationships from real molecular substructures. Empirical results demonstrate that MO_SCPLOWOLC_SCPLOW-AE achieves a large margin performance gain compared to the current state-of-the-art 3D molecular modeling approach. The source codes of MO_SCPLOWOLC_SCPLOW-AE are publicly available at https://github.com/yjwtheonly/MolAE.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Cross-Modality and Self-Supervised Protein Embedding for Compound-Protein Affinity and Contact Prediction 97%
- Neural Collective Matrix Factorization for Integrated Analysis of Heterogeneous Biomedical Data 95%
- DTI-Voodoo: machine learning over interaction networks and ontology-based background knowledge predicts drug-target interactions 95%
Similar papers in this journal
Similar papers in this journal
- FLONE: fully Lorentz network embedding for inferring novel drug targets 96%
- MULAN: Multimodal Protein Language Model for Sequence and Structure Encoding 95%
- SAINT-Angle: self-attention augmented inception-inside-inception network and transfer learning improve protein backbone torsion angle prediction 93%
Similar papers in this journal
- All-Atom Protein Sequence Design using Discrete Diffusion Models 95%
- DeepGraphMol, a multi-objective, computational strategy for generating molecules with desirable properties: a graph convolution and reinforcement learning approach 95%
- AdapToR: Adaptive Topological Regression for quantitative structure-activity relationship modeling 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.