Knowledge graphs generally suffer from incompleteness, which can be
alleviated by completing the missing information. Deep knowledge convolutional
embedding models based on neural networks are currently popular methods for
knowledge graph completion. However, most existing methods use external
convolution kernels and traditional plain convolution processes, which limits
the feature interaction capability of the model. In this paper, we propose a
novel dynamic convolutional embedding model ConvD for knowledge graph
completion, which directly reshapes the relation embeddings into multiple
internal convolution kernels to improve the external convolution kernels of the
traditional convolutional embedding model. The internal convolution kernels can
effectively augment the feature interaction between the relation embeddings and
entity embeddings, thus enhancing the model embedding performance. Moreover, we
design a priori knowledge-optimized attention mechanism, which can assign
different contribution weight coefficients to multiple relation convolution
kernels for dynamic convolution to improve the expressiveness of the model
further. Extensive experiments on various datasets show that our proposed model
consistently outperforms the state-of-the-art baseline methods, with average
improvements ranging from 11.30\% to 16.92\% across all model evaluation
metrics. Ablation experiments verify the effectiveness of each component module
of the ConvD model.

本文提出了一种新颖的动态卷积嵌入模型 ConvD，用于知识图谱补全，并通过将关系嵌入重塑为多个内部卷积核，改善传统卷积嵌入模型的外部卷积核，有效增强了关系嵌入与实体嵌入之间的特征交互，进一步提高了模型的表达能力。通过广泛的实验证明，我们提出的模型在各种数据集上始终优于最先进的基线方法，平均改进范围从 11.30% 到 16.92%。消融实验证实了 ConvD 模型的每个组件模块的有效性。

ConvD：增强动态卷积嵌入的知识图谱补全

ConvD: Attention Enhanced Dynamic Convolutional Embeddings for Knowledge  Graph Completion

This paper introduces TRACE-GPT, which stands for Time-seRies
Anomaly-detection with Convolutional Embedding and Generative Pre-trained
Transformers. TRACE-GPT is designed to pre-train univariate time-series sensor
data and detect faults on unlabeled datasets in semiconductor manufacturing. In
semiconductor industry, classifying abnormal time-series sensor data from
normal data is important because it is directly related to wafer defect.
However, small, unlabeled, and even mixed training data without enough
anomalies make classification tasks difficult. In this research, we capture
features of time-series data with temporal convolutional embedding and
Generative Pre-trained Transformer (GPT) to classify abnormal sequences from
normal sequences using cross entropy loss. We prove that our model shows better
performance than previous unsupervised models with both an open dataset, the
University of California Riverside (UCR) time-series classification archive,
and the process log of our Chemical Vapor Deposition (CVD) equipment. Our model
has the highest F1 score at Equal Error Rate (EER) across all datasets and is
only 0.026 below the supervised state-of-the-art baseline on the open dataset.

TRACE-GPT 通过卷积嵌入和生成式预训练转换器，用于预训练单变量时间序列传感器数据，并检测半导体制造中的未标记数据中的故障。

半导体制造无监督故障检测的时间序列数据生成预训练

Generative Pre-Training of Time-Series Data for Unsupervised Fault  Detection in Semiconductor Manufacturing

Vision Transformers (ViTs) have recently dominated a range of computer vision
tasks, yet it suffers from low training data efficiency and inferior local
semantic representation capability without appropriate inductive bias.
Convolutional neural networks (CNNs) inherently capture regional-aware
semantics, inspiring researchers to introduce CNNs back into the architecture
of the ViTs to provide desirable inductive bias for ViTs. However, is the
locality achieved by the micro-level CNNs embedded in ViTs good enough? In this
paper, we investigate the problem by profoundly exploring how the macro
architecture of the hybrid CNNs/ViTs enhances the performances of hierarchical
ViTs. Particularly, we study the role of token embedding layers, alias
convolutional embedding (CE), and systemically reveal how CE injects desirable
inductive bias in ViTs. Besides, we apply the optimal CE configuration to 4
recently released state-of-the-art ViTs, effectively boosting the corresponding
performances. Finally, a family of efficient hybrid CNNs/ViTs, dubbed CETNets,
are released, which may serve as generic vision backbones. Specifically,
CETNets achieve 84.9% Top-1 accuracy on ImageNet-1K (training from scratch),
48.6% box mAP on the COCO benchmark, and 51.6% mIoU on the ADE20K,
substantially improving the performances of the corresponding state-of-the-art
baselines.

本文研究了如何使用混合 CNN / ViTs 的宏观架构来增强分层 ViTs 的性能，特别地，我们研究了令牌嵌入层的作用，引入了卷积嵌入（CE），并系统地揭示了 CE 如何在 ViTs 中注入理想的归纳偏差。 我们将最佳 CE 配置应用于 4 个最近发布的最先进的 ViT 中，有效地提高了相应的性能，并释放了一组高效的混合 CNN / ViT，名为 CETNets，可作为通用视觉骨干。