Action in video usually involves the interaction of human with objects.
Action labels are typically composed of various combinations of verbs and
nouns, but we may not have training data for all possible combinations. In this
paper, we aim to improve the generalization ability of the compositional action
recognition model to novel verbs or novel nouns that are unseen during training
time, by leveraging the power of knowledge graphs. Previous work utilizes
verb-noun compositional action nodes in the knowledge graph, making it
inefficient to scale since the number of compositional action nodes grows
quadratically with respect to the number of verbs and nouns. To address this
issue, we propose our approach: Disentangled Action Recognition with
Knowledge-bases (DARK), which leverages the inherent compositionality of
actions. DARK trains a factorized model by first extracting disentangled
feature representations for verbs and nouns, and then predicting classification
weights using relations in external knowledge graphs. The type constraint
between verb and noun is extracted from external knowledge bases and finally
applied when composing actions. DARK has better scalability in the number of
objects and verbs, and achieves state-of-the-art performance on the Charades
dataset. We further propose a new benchmark split based on the Epic-kitchen
dataset which is an order of magnitude bigger in the numbers of classes and
samples, and benchmark various models on this benchmark.

本研究提出了一种使用知识图谱来改善组成动作识别模型对于新动词或新名词的泛化能力的方法，通过提取动词和名词的不可分离特征表示并利用外部知识图谱中的关系来预测分类权重构成动作，并取得了 Charades 数据集上的最佳表现。

基于知识库的解耦合动作识别

Disentangled Action Recognition with Knowledge Bases

We show that for human-object interaction detection a relatively simple
factorized model with appearance and layout encodings constructed from
pre-trained object detectors outperforms more sophisticated approaches. Our
model includes factors for detection scores, human and object appearance, and
coarse (box-pair configuration) and optionally fine-grained layout (human
pose). We also develop training techniques that improve learning efficiency by:
(1) eliminating a train-inference mismatch; (2) rejecting easy negatives during
mini-batch training; and (3) using a ratio of negatives to positives that is
two orders of magnitude larger than existing approaches. We conduct a thorough
ablation study to understand the importance of different factors and training
techniques using the challenging HICO-Det dataset.

本研究发现，在人物 - 物体交互检测方面，使用预训练的物体检测器构建的外观和布局编码的相对简单的分解模型优于更复杂的方法。同时，作者还开发了一些训练技巧来提高学习效率。研究使用现有的 HICO-Det 数据集展开了深入的剖析，以了解不同因素和培训技术的重要性。

简陋版人 - 物互动检测：分解、布局编码和训练技术

No-Frills Human-Object Interaction Detection: Factorization, Layout  Encodings, and Training Techniques

Retrieval tasks typically require a ranking of items given a query.
Collaborative filtering tasks, on the other hand, learn to model user's
preferences over items. In this paper we study the joint problem of
recommending items to a user with respect to a given query, which is a
surprisingly common task. This setup differs from the standard collaborative
filtering one in that we are given a query x user x item tensor for training
instead of the more traditional user x item matrix. Compared to document
retrieval we do have a query, but we may or may not have content features (we
will consider both cases) and we can also take account of the user's profile.
We introduce a factorized model for this new task that optimizes the top-ranked
items returned for the given query and user. We report empirical results where
it outperforms several baselines.

本研究介绍了一种基于矩阵因式分解的推荐模型，用于针对给定的查询和用户，推荐排名靠前的物品，实验结果表明它胜过了多个基准模型。