Controllable Protein Design by Prefix-Tuning Protein Language Models

The design of novel proteins with tailored functionalities, particularly in drug discovery and vaccine development, presents a transformative approach to addressing pressing biomedical challenges. Inspired by the remarkable success of pre-trained language models in natural language processing (NLP), protein language models (ProtLMs) have emerged as powerful tools in advancing protein science. While NLP leverages flexible text-based control tags to prompt language model generation, the restricted amino acid space (limited to 20 residues) imposes inherent constraints on achieving analogous controllability. In this study, we propose PrefixProt, a framework for controllable protein design that employs prefix tuning to learn virtual tokens as control tags. These virtual tokens are adaptively tailored to diverse protein properties through a data-driven manner and can be combinatorially integrated to enable multi-objective control over protein generation. The effectiveness of PrefixProt was validated through extensive experiments encompassing both protein structure design (e.g. alpha-helix or beta-sheet topologies) and protein function design (e.g. antimicrobial or anticancer peptide activities). Benchmark results demonstrate that prefix virtual tokens efficiently guide the pre-trained ProtLM by optimizing a smaller number of trainable parameters, out-performing other parameter-efficient fine-tuning methods and text-guided ProtLMs, particularly in scenarios with limited data availability. More importantly, the compositional flexibility of virtual tokens facilitates the generation of proteins with multiple target properties, substantially expanding the scope of design possibilities. By harmonizing controllability, efficiency and generalizability, PrefixProt establishes a robust framework for de novo protein design, with promising applications in drug discovery and biomedicine. Availability and implementationThe models and code are available at: https://github.com/chen-bioinfo/PrefixProt