The human immune response depends on the binding of T-cell receptors (TCRs)
to antigens (pTCR), which elicits the T cells to eliminate viruses, tumor
cells, and other pathogens. The ability of human immunity system responding to
unknown viruses and bacteria stems from the TCR diversity. However, this vast
diversity poses challenges on the TCR-antigen binding prediction methods. In
this study, we propose a Masked Language Model (MLM), referred to as tcrLM, to
overcome limitations in model generalization. Specifically, we randomly masked
sequence segments and train tcrLM to infer the masked segment, thereby extract
expressive feature from TCR sequences. Meanwhile, we introduced virtual
adversarial training techniques to enhance the model's robustness. We built the
largest TCR CDR3 sequence dataset to date (comprising 2,277,773,840 residuals),
and pre-trained tcrLM on this dataset. Our extensive experimental results
demonstrate that tcrLM achieved AUC values of 0.937 and 0.933 on independent
test sets and external validation sets, respectively, which remarkably
outperformed four previously published prediction methods. On a large-scale
COVID-19 pTCR binding test set, our method outperforms the current
state-of-the-art method by at least 8%, highlighting the generalizability of
our method. Furthermore, we validated that our approach effectively predicts
immunotherapy response and clinical outcomes on a clinical cohorts. These
findings clearly indicate that tcrLM exhibits significant potential in
predicting antigenic immunogenicity.

我们提出了一种蒙版语言模型（tcrLM），用于预测 T 细胞受体（TCR）与抗原的结合，通过训练 tcrLM 来推断被屏蔽的片段，从 TCR 序列中提取表达特征，并在大规模 COVID-19 pTCR 结合测试集上超过当前最先进的方法，有效预测免疫疗法反应和临床结果。

用于预测 T 细胞受体 - 抗原结合特异性的大型语言模型

A large language model for predicting T cell receptor-antigen binding  specificity

An accurate binding affinity prediction between T-cell receptors and epitopes
contributes decisively to develop successful immunotherapy strategies. Some
state-of-the-art computational methods implement deep learning techniques by
integrating evolutionary features to convert the amino acid residues of cell
receptors and epitope sequences into numerical values, while some other methods
employ pre-trained language models to summarize the embedding vectors at the
amino acid residue level to obtain sequence-wise representations.
Here, we propose a highly reliable novel method, MATE-Pred, that performs
multi-modal attention-based prediction of T-cell receptors and epitopes binding
affinity. The MATE-Pred is compared and benchmarked with other deep learning
models that leverage multi-modal representations of T-cell receptors and
epitopes. In the proposed method, the textual representation of proteins is
embedded with a pre-trained bi-directional encoder model and combined with two
additional modalities: a) a comprehensive set of selected physicochemical
properties; b) predicted contact maps that estimate the 3D distances between
amino acid residues in the sequences.
The MATE-Pred demonstrates the potential of multi-modal model in achieving
state-of-the-art performance (+8.4\% MCC, +5.5\% AUC compared to baselines) and
efficiently capturing contextual, physicochemical, and structural information
from amino acid residues. The performance of MATE-Pred projects its potential
application in various drug discovery regimes.

MATE-Pred 是一种高度可靠的新方法，通过多模式注意力机制预测 T 细胞受体和表位结合亲和力，展示了多模式模型在从氨基酸残基中高效捕捉上下文、物理化学和结构信息的潜力，以及在各种药物发现方案中的潜在应用。