This paper describes our multiclass classification system developed as part
of the LTEDI@RANLP-2023 shared task. We used a BERT-based language model to
detect homophobic and transphobic content in social media comments across five
language conditions: English, Spanish, Hindi, Malayalam, and Tamil. We
retrained a transformer-based crosslanguage pretrained language model,
XLMRoBERTa, with spatially and temporally relevant social media language data.
We also retrained a subset of models with simulated script-mixed social media
language data with varied performance. We developed the best performing
seven-label classification system for Malayalam based on weighted macro
averaged F1 score (ranked first out of six) with variable performance for other
language and class-label conditions. We found the inclusion of this
spatio-temporal data improved the classification performance for all language
and task conditions when compared with the baseline. The results suggests that
transformer-based language classification systems are sensitive to
register-specific and language-specific retraining.

该研究使用 BERT-based 语言模型开发了多类别分类系统，用于检测社交媒体评论中的恐同和恐 Trans 内容，跨五种语言条件：英语，西班牙语，印地语，马拉雅拉姆语和泰米尔语。发现使用时空相关的社交媒体语言数据可以提高语言分类系统的性能。

使用时空再训练的语言模型检测社交媒体评论中的恐同 / 恐跨性别现象

cantnlp@LT-EDI@RANLP-2023: Homophobia/Transphobia Detection in Social  Media Comments using Spatio-Temporally Retrained Language Models

Extracting temporal relationships between pairs of events in texts is a
crucial yet challenging problem for natural language understanding. Depending
on the distance between the events, models must learn to differently balance
information from local and global contexts surrounding the event pair for
temporal relation prediction. Learning how to fuse this information has proved
challenging for transformer-based language models. Therefore, we present MulCo:
Multi-Scale Contrastive Co-Training, a technique for the better fusion of local
and global contextualized features. Our model uses a BERT-based language model
to encode local context and a Graph Neural Network (GNN) to represent global
document-level syntactic and temporal characteristics. Unlike previous
state-of-the-art methods, which use simple concatenation on multi-view features
or select optimal sentences using sophisticated reinforcement learning
approaches, our model co-trains GNN and BERT modules using a multi-scale
contrastive learning objective. The GNN and BERT modules learn a synergistic
parameterization by contrasting GNN multi-layer multi-hop subgraphs (i.e.,
global context embeddings) and BERT outputs (i.e., local context embeddings)
through end-to-end back-propagation. We empirically demonstrate that MulCo
provides improved ability to fuse local and global contexts encoded using BERT
and GNN compared to the current state-of-the-art. Our experimental results show
that MulCo achieves new state-of-the-art results on several temporal relation
extraction datasets.

MulCo 是一种用于更好地融合 BERT 和 GNN 对本地和全局上下文特征的技术，通过对比 GNN 多层多跳子图（全局上下文嵌入）和 BERT 输出（本地上下文嵌入），MulCo 实现了 GNN 和 BERT 模块的协同参数化。MulCo 在多个时间关系提取数据集上取得了新的最先进结果。

多尺度对比协同训练用于事件时间关系抽取

Multi-Scale Contrastive Co-Training for Event Temporal Relation Extraction

The UMLS Metathesaurus integrates more than 200 biomedical source
vocabularies. During the Metathesaurus construction process, synonymous terms
are clustered into concepts by human editors, assisted by lexical similarity
algorithms. This process is error-prone and time-consuming. Recently, a deep
learning model (LexLM) has been developed for the UMLS Vocabulary Alignment
(UVA) task. This work introduces UBERT, a BERT-based language model, pretrained
on UMLS terms via a supervised Synonymy Prediction (SP) task replacing the
original Next Sentence Prediction (NSP) task. The effectiveness of UBERT for
UMLS Metathesaurus construction process is evaluated using the UMLS Vocabulary
Alignment (UVA) task. We show that UBERT outperforms the LexLM, as well as
biomedical BERT-based models. Key to the performance of UBERT are the synonymy
prediction task specifically developed for UBERT, the tight alignment of
training data to the UVA task, and the similarity of the models used for
pretrained UBERT.

介绍了 UBERT，一种基于 BERT 的语言模型，通过替换原始的下一句预测任务为监督的同义预测任务进行 UMLS 术语的预训练，该模型能有效地在 UMLS Metathesaurus 的构建过程中替代 LexLM 模型，并且对超过 200 种医学基础词汇进行聚类并识别其中的同义词。