Self-attention transformers have demonstrated accuracy for image
classification with smaller data sets. However, a limitation is that tests
to-date are based upon single class image detection with known representation
of image populations. For instances where the input image classes may be
greater than one and test sets that lack full information on representation of
image populations, accuracy calculations must adapt. The Receiver Operating
Characteristic (ROC) accuracy thresh-old can address the instances of
multi-class input images. However, this approach is unsuitable in instances
where image population representation is unknown. We consider calculating
accuracy using the knee method to determine threshold values on an ad-hoc
basis. Results of ROC curve and knee thresholds for a multi-class data set,
created from CIFAR-10 images, are discussed for multi-class image detection.

基于 CIFAR-10 图像创建的多类图像检测数据集讨论了使用自注意力变换器对图像分类中的多类输入图像进行准确性计算的 ROC 曲线和 Knee 方法阈值的结果。

膝盖或 ROC

Knee or ROC

A demonstration of a real-time and continuous turn-taking prediction system
is presented. The system is based on a voice activity projection (VAP) model,
which directly maps dialogue stereo audio to future voice activities. The VAP
model includes contrastive predictive coding (CPC) and self-attention
transformers, followed by a cross-attention transformer. We examine the effect
of the input context audio length and demonstrate that the proposed system can
operate in real-time with CPU settings, with minimal performance degradation.

提出了一个基于语音活动投影（VAP）模型的实时连续交替预测系统演示。该系统基于对话立体声音频直接预测未来语音活动。VAP 模型包括对比预测编码（CPC）和自注意力变换器，之后是交叉注意力变换器。我们考察了输入上下文音频长度的影响，并演示了该系统可以在 CPU 设置下以实时方式运行，并具有最小的性能降低。

实时连续的话语交替预测：使用声音活动投影

Real-time and Continuous Turn-taking Prediction Using Voice Activity  Projection

Explainability techniques are crucial in gaining insights into the reasons
behind the predictions of deep learning models, which have not yet been applied
to chemical language models. We propose an explainable AI technique that
attributes the importance of individual atoms towards the predictions made by
these models. Our method backpropagates the relevance information towards the
chemical input string and visualizes the importance of individual atoms. We
focus on self-attention Transformers operating on molecular string
representations and leverage a pretrained encoder for finetuning. We showcase
the method by predicting and visualizing solubility in water and organic
solvents. We achieve competitive model performance while obtaining
interpretable predictions, which we use to inspect the pretrained model.

该研究提出了一种可解释的 AI 技术来解释和预测化学模型，能够精确推断分子溶度和物理属性。

化学语言模型的可解释性技术

Explainability Techniques for Chemical Language Models

A fundamental question in natural language processing is - what kind of
language structure and semantics is the language model capturing? Graph formats
such as knowledge graphs are easy to evaluate as they explicitly express
language semantics and structure. This study evaluates the semantics encoded in
the self-attention transformers by leveraging explicit knowledge graph
structures. We propose novel metrics to measure the reconstruction error when
providing graph path sequences from a knowledge graph and trying to
reproduce/reconstruct the same from the outputs of the self-attention
transformer models. The opacity of language models has an immense bearing on
societal issues of trust and explainable decision outcomes. Our findings
suggest that language models are models of stochastic control processes for
plausible language pattern generation. However, they do not ascribe object and
concept-level meaning and semantics to the learned stochastic patterns such as
those described in knowledge graphs. Furthermore, to enable robust evaluation
of concept understanding by language models, we construct and make public an
augmented language understanding benchmark built on the General Language
Understanding Evaluation (GLUE) benchmark. This has significant
application-level user trust implications as stochastic patterns without a
strong sense of meaning cannot be trusted in high-stakes applications.

本研究通过建立图格式的知识图谱，对自注意力转换器编码的语义进行了评估，发现语言模型不会赋予学习到的随机模式以物体和概念级别的含义和语义，同时构建了一个 GLUE 基准的增强语言理解基准，以使语言模型的概念理解得到健壮的评估。