Personalized text generation presents a specialized mechanism for delivering
content that is specific to a user's personal context. While the research
progress in this area has been rapid, evaluation still presents a challenge.
Traditional automated metrics such as BLEU and ROUGE primarily measure lexical
similarity to human-written references, and are not able to distinguish
personalization from other subtle semantic aspects, thus falling short of
capturing the nuances of personalized generated content quality. On the other
hand, human judgments are costly to obtain, especially in the realm of
personalized evaluation. Inspired by these challenges, we explore the use of
large language models (LLMs) for evaluating personalized text generation, and
examine their ability to understand nuanced user context. We present AuPEL, a
novel evaluation method that distills three major semantic aspects of the
generated text: personalization, quality and relevance, and automatically
measures these aspects. To validate the effectiveness of AuPEL, we design
carefully controlled experiments and compare the accuracy of the evaluation
judgments made by LLMs versus that of judgements made by human annotators, and
conduct rigorous analyses of the consistency and sensitivity of the proposed
metric. We find that, compared to existing evaluation metrics, AuPEL not only
distinguishes and ranks models based on their personalization abilities more
accurately, but also presents commendable consistency and efficiency for this
task. Our work suggests that using LLMs as the evaluators of personalized text
generation is superior to traditional text similarity metrics, even though
interesting new challenges still remain.

个性化文本生成采用了一种专门的机制来提供与用户个人背景相关的内容，并且尽管这一领域的研究进展很快，但仍存在着评估方面的挑战。本研究提出了 AuPEL 作为一种新颖的评估方法，通过大型语言模型对个性化文本生成进行评估，用以满足人工评估的高成本和传统文本相似度度量方法的不足。实验证明，与现有的评估指标相比，AuPEL 不仅可以更准确地区分和排名模型的个性化能力，而且在此任务中表现出令人称赞的一致性和高效性。

基于大型语言模型的个性化文本生成的自动评估

Automated Evaluation of Personalized Text Generation using Large  Language Models

Knowledge graphs represent real-world entities and their relations in a
semantically-rich structure supported by ontologies. Exploring this data with
machine learning methods often relies on knowledge graph embeddings, which
produce latent representations of entities that preserve structural and local
graph neighbourhood properties, but sacrifice explainability. However, in tasks
such as link or relation prediction, understanding which specific features
better explain a relation is crucial to support complex or critical
applications.
We propose SEEK, a novel approach for explainable representations to support
relation prediction in knowledge graphs. It is based on identifying relevant
shared semantic aspects (i.e., subgraphs) between entities and learning
representations for each subgraph, producing a multi-faceted and explainable
representation.
We evaluate SEEK on two real-world highly complex relation prediction tasks:
protein-protein interaction prediction and gene-disease association prediction.
Our extensive analysis using established benchmarks demonstrates that SEEK
achieves significantly better performance than standard learning representation
methods while identifying both sufficient and necessary explanations based on
shared semantic aspects.

该研究提出了一种基于语义相关性子图学习的可解释性表示方法 SEEK，能够提高知识图谱关系预测的性能和解释性。研究评估了 SEEK 在复杂关系预测任务上的表现，并表明了其优于标准学习表示方法。

知识图谱中关系预测的可解释表示

Explainable Representations for Relation Prediction in Knowledge Graphs

Video recognition has been advanced in recent years by benchmarks with rich
annotations. However, research is still mainly limited to human action or
sports recognition - focusing on a highly specific video understanding task and
thus leaving a significant gap towards describing the overall content of a
video. We fill this gap by presenting a large-scale "Holistic Video
Understanding Dataset"~(HVU). HVU is organized hierarchically in a semantic
taxonomy that focuses on multi-label and multi-task video understanding as a
comprehensive problem that encompasses the recognition of multiple semantic
aspects in the dynamic scene. HVU contains approx.~572k videos in total with 9
million annotations for training, validation, and test set spanning over 3142
labels. HVU encompasses semantic aspects defined on categories of scenes,
objects, actions, events, attributes, and concepts which naturally captures the
real-world scenarios.
We demonstrate the generalization capability of HVU on three challenging
tasks: 1.) Video classification, 2.) Video captioning and 3.) Video clustering
tasks. In particular for video classification, we introduce a new
spatio-temporal deep neural network architecture called "Holistic Appearance
and Temporal Network"~(HATNet) that builds on fusing 2D and 3D architectures
into one by combining intermediate representations of appearance and temporal
cues. HATNet focuses on the multi-label and multi-task learning problem and is
trained in an end-to-end manner. Via our experiments, we validate the idea that
holistic representation learning is complementary, and can play a key role in
enabling many real-world applications.

本文介绍了一个基于语义分类的大规模 Holistic 视频理解数据集 HVU，它实现了对多标签、多任务视频理解的全面考虑，以及介绍了一种新的综合外观和时间神经网络体系结构 HATNet，本文拓展了视频识别范畴，验证了 Holistic 表示学习在许多实际应用程序中的重要作用。