Human language acquisition is an efficient, supervised, and continual
process. In this work, we took inspiration from how human babies acquire their
first language, and developed a computational process for word acquisition
through comparative learning. Motivated by cognitive findings, we generated a
small dataset that enables the computation models to compare the similarities
and differences of various attributes, learn to filter out and extract the
common information for each shared linguistic label. We frame the acquisition
of words as not only the information filtration process, but also as
representation-symbol mapping. This procedure does not involve a fixed
vocabulary size, nor a discriminative objective, and allows the models to
continually learn more concepts efficiently. Our results in controlled
experiments have shown the potential of this approach for efficient continual
learning of grounded words.

通过比较学习的计算过程，本研究模拟了人类婴儿习得第一门语言的方式，开发了一种用于词汇习得的计算过程，通过筛选和提取共享语言标签的共同信息，并将词汇的习得过程视为信息过滤和符号映射，以实现有效的持续学习。在对照实验中得到的结果显示了这种方法在词汇习得方面的潜力。

人类启发的渐进对齐与比较学习 用于基于语境的词汇习得

Human Inspired Progressive Alignment and Comparative Learning for  Grounded Word Acquisition

Traditional comparative learning sentence embedding directly uses the encoder
to extract sentence features, and then passes in the comparative loss function
for learning. However, this method pays too much attention to the sentence body
and ignores the influence of some words in the sentence on the sentence
semantics. To this end, we propose CMLM-CSE, an unsupervised contrastive
learning framework based on conditional MLM. On the basis of traditional
contrastive learning, an additional auxiliary network is added to integrate
sentence embedding to perform MLM tasks, forcing sentence embedding to learn
more masked word information. Finally, when Bertbase was used as the
pretraining language model, we exceeded SimCSE by 0.55 percentage points on
average in textual similarity tasks, and when Robertabase was used as the
pretraining language model, we exceeded SimCSE by 0.3 percentage points on
average in textual similarity tasks.

提出了一种基于条件 MLM 的无监督对比学习框架 CMLM-CSE，通过增加辅助网络来集成句子嵌入执行 MLM 任务，使句子嵌入学习更多的屏蔽词信息，使用 Bertbase 和 Robertabase 作为预训练语言模型进行预训练，在文本相似性任务中分别超过了 SimCSE 0.55 和 0.3 个百分点。

基于条件 MLM 对比学习的句子嵌入方法 (CMLM-CSE)

CMLM-CSE: Based on Conditional MLM Contrastive Learning for Sentence  Embeddings

The challenges faced by text classification with large tag systems in natural
language processing tasks include multiple tag systems, uneven data
distribution, and high noise. To address these problems, the ESimCSE
unsupervised comparative learning and UDA semi-supervised comparative learning
models are combined through the use of joint training techniques in the
models.The ESimCSE model efficiently learns text vector representations using
unlabeled data to achieve better classification results, while UDA is trained
using unlabeled data through semi-supervised learning methods to improve the
prediction performance of the models and stability, and further improve the
generalization ability of the model. In addition, adversarial training
techniques FGM and PGD are used in the model training process to improve the
robustness and reliability of the model. The experimental results show that
there is an 8% and 10% accuracy improvement relative to Baseline on the public
dataset Ruesters as well as on the operational dataset, respectively, and a 15%
improvement in manual validation accuracy can be achieved on the operational
dataset, indicating that the method is effective.

研究采用 ESimCSE 和 UDA 模型相结合，利用联合训练方法提高模型分类效果，通过半监督学习方法和对抗训练技术提高模型稳定性和鲁棒性，实验证明在文本分类领域取得显著成果。