Spontaneous Emergence of Symmetry in a Generative Model of Protein Structure
Oresten, A.; Sato, K.; Stalmarck, A.; Billera, L.; Nordlinder, H. N.; Ryder, J. C.; Kaduk, M.; Murrell, B.
Show abstract
Generative models are becoming powerful tools for protein design, enabling the creation of novel protein structures and sequences. Recent approaches have shown success using diffusion models and flow matching to sample realistic protein folds. Many methods explicitly incorporate structural biases or constraints--for example, enforcing symmetry during generation--to steer the design process. Here we report the spontaneous emergence of structural symmetry in a transformer-based generative model for proteins, without any symmetry-specific conditioning during training or constraints during generation. In our flow-matching model, a single attention head in an SE(3)-equivariant transformer layer was found to be primarily responsible for the models ability to generate symmetric arrangements of backbone residues across chains, or repeating motifs within a chain. Our results show that protein generative models can learn high-level structural patterns implicitly from training data. This opens new questions about interpretability and control in generative design: understanding how and why a single attention head can govern a complex global property like symmetry may inform future model architectures and help exploit emergent behaviors for better protein engineering.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Controllable Protein Design via Autoregressive Direct Coupling Analysis Conditioned on Principal Components 97%
- BAGEL: Protein Engineering via Exploration of an Energy Landscape 97%
- Computational design of novel Cas9 PAM-interacting domains using evolution-based modelling and structural quality assessment 96%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.