Generative modeling of short, disordered proteins with homogeneous sequence composition

Protein design has seen remarkable progress in the past decade, with numerous examples of de novo proteins with novel topologies and functions being successfully synthesized. Computational tools have played a large role in the ability to rationally design proteins. Recently, there have been numerous successes applying deep learning techniques to protein design that have demonstrated comparable or significantly improved performance over traditional energy-based approaches. However, the protein design problem has been less well explored for disordered proteins or, more generally, proteins with conformational heterogeneity. In this work, we demonstrate that if one approximates the spatial output of a coarse-grained molecular dynamics simulation as a multivariate normal distribution parameterized by a mean vector (representing an ensemble-averaged pairwise distance map) and covariance matrix, one can train a generative model to learn the distribution of these parameters across a set of sequences. Specifically, we encoded the mean vector and covariance matrix for each sequence in a low-dimensional space via a fixed linear transformation and trained a masked auto-encoder to accurately learn the distribution of this low-dimensional output. Furthermore, by sampling from the masked auto-encoder and transforming the generated samples back into their original high-dimensional space, one can generate realistic, ensemble-averaged pairwise distance maps. These results were demonstrated on coarse-grained simulation data derived from approximately 2000 distinct sequences, each sequence being 24 residues in length and consisting exclusively of glycine, serine, glutamate, and lysine. Though this set of sequences is relatively homogeneous in composition, we speculate our approach can be applied to disordered sequences of longer length and more heterogeneous composition, given the appropriate training set.