Because most of the scientific literature data is unmarked, it makes semantic
representation learning based on unsupervised graph become crucial. At the same
time, in order to enrich the features of scientific literature, a learning
method of semantic representation of scientific literature based on adaptive
features and graph neural network is proposed. By introducing the adaptive
feature method, the features of scientific literature are considered globally
and locally. The graph attention mechanism is used to sum the features of
scientific literature with citation relationship, and give each scientific
literature different feature weights, so as to better express the correlation
between the features of different scientific literature. In addition, an
unsupervised graph neural network semantic representation learning method is
proposed. By comparing the mutual information between the positive and negative
local semantic representation of scientific literature and the global graph
semantic representation in the potential space, the graph neural network can
capture the local and global information, thus improving the learning ability
of the semantic representation of scientific literature. The experimental
results show that the proposed learning method of semantic representation of
scientific literature based on adaptive feature and graph neural network is
competitive on the basis of scientific literature classification, and has
achieved good results.

提出了一种基于自适应特征和图神经网络的科学文献语义表示学习方法，通过引入自适应特征方法，综合考虑了科学文献的全局和局部特征，借助图注意机制对具有引文关系的科学文献的特征进行加权求和，进一步表达不同科学文献特征之间的相关性，同时通过无监督图神经网络学习方法，提高了科学文献的语义表示学习能力，实验结果表明该方法在科学文献分类方面具有竞争力并取得了良好结果。

基于自适应特征和图神经网络的科学文献语义表示学习

Semantic Representation Learning of Scientific Literature based on  Adaptive Feature and Graph Neural Network

Story generation aims to generate a long narrative conditioned on a given
input. In spite of the success of prior works with the application of
pre-trained models, current neural models for Chinese stories still struggle to
generate high-quality long text narratives. We hypothesise that this stems from
ambiguity in syntactically parsing the Chinese language, which does not have
explicit delimiters for word segmentation. Consequently, neural models suffer
from the inefficient capturing of features in Chinese narratives. In this
paper, we present a new generation framework that enhances the feature
capturing mechanism by informing the generation model of dependencies between
words and additionally augmenting the semantic representation learning through
synonym denoising training. We conduct a range of experiments, and the results
demonstrate that our framework outperforms the state-of-the-art Chinese
generation models on all evaluation metrics, demonstrating the benefits of
enhanced dependency and semantic representation learning.

本文研究生成长篇叙述性内容的故事生成问题，提出了一种新的生成框架，通过依存关系增强机制和同义词去噪训练增强语义表示学习，显著提高了中文故事生成模型性能。

通过语法依赖和语义意识提高中文故事生成

Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics

Semantic representation learning for sentences is an important and
well-studied problem in NLP. The current trend for this task involves training
a Transformer-based sentence encoder through a contrastive objective with text,
i.e., clustering sentences with semantically similar meanings and scattering
others. In this work, we find the performance of Transformer models as sentence
encoders can be improved by training with multi-modal multi-task losses, using
unpaired examples from another modality (e.g., sentences and unrelated
image/audio data). In particular, besides learning by the contrastive loss on
text, our model clusters examples from a non-linguistic domain (e.g.,
visual/audio) with a similar contrastive loss at the same time. The reliance of
our framework on unpaired non-linguistic data makes it language-agnostic,
enabling it to be widely applicable beyond English NLP. Experiments on 7
semantic textual similarity benchmarks reveal that models trained with the
additional non-linguistic (images/audio) contrastive objective lead to higher
quality sentence embeddings. This indicates that Transformer models are able to
generalize better by doing a similar task (i.e., clustering) with unpaired
examples from different modalities in a multi-task fashion.

本文提出了一种多模态多任务的 Transformer 模型，通过对句子和非语言数据对比学习的方式来提高句子编码器的性能，并在基准测试中获得了更高质量的语义文本相似性结果。