Antibodies are proteins produced by the immune system that can identify and
neutralise a wide variety of antigens with high specificity and affinity, and
constitute the most successful class of biotherapeutics. With the advent of
next-generation sequencing, billions of antibody sequences have been collected
in recent years, though their application in the design of better therapeutics
has been constrained by the sheer volume and complexity of the data. To address
this challenge, we present IgBert and IgT5, the best performing
antibody-specific language models developed to date which can consistently
handle both paired and unpaired variable region sequences as input. These
models are trained comprehensively using the more than two billion unpaired
sequences and two million paired sequences of light and heavy chains present in
the Observed Antibody Space dataset. We show that our models outperform
existing antibody and protein language models on a diverse range of design and
regression tasks relevant to antibody engineering. This advancement marks a
significant leap forward in leveraging machine learning, large scale data sets
and high-performance computing for enhancing antibody design for therapeutic
development.

我们提出了 IgBert 和 IgT5，这两个迄今为止最好的抗体特异性语言模型，在设计更好的治疗方法方面，能持续处理成对和不成对的变量区域序列作为输入，并且在抗体工程的多样化设计和回归任务上，这些模型胜过了现有的抗体和蛋白质语言模型。这一进展在利用机器学习、大规模数据集和高性能计算增强抗体治疗方法的设计方面迈出了重要的一步。