We introduce a simple method that employs pre-trained CLIP encoders to
enhance model generalization in the ALFRED task. In contrast to previous
literature where CLIP replaces the visual encoder, we suggest using CLIP as an
additional module through an auxiliary object detection objective. We validate
our method on the recently proposed Episodic Transformer architecture and
demonstrate that incorporating CLIP improves task performance on the unseen
validation set. Additionally, our analysis results support that CLIP especially
helps with leveraging object descriptions, detecting small objects, and
interpreting rare words.

使用预训练的 CLIP 编码器通过辅助目标检测目标，在 Episodic Transformer 架构中整合 CLIP 可以提高 ALFRED 任务的性能，并且尤其有助于利用物体描述、检测小物体和解释罕见词汇。

面对未知环境的常见物体错误：ET tu, CLIP?

ET tu, CLIP? Addressing Common Object Errors for Unseen Environments

A fundamental requirement of any task-oriented dialogue system is the ability
to generate object descriptions that refer to objects in the task domain. The
subproblem of content selection for object descriptions in task-oriented
dialogue has been the focus of much previous work and a large number of models
have been proposed. In this paper, we use the annotated COCONUT corpus of
task-oriented design dialogues to develop feature sets based on Dale and
Reiters (1995) incremental model, Brennan and Clarks (1996) conceptual pact
model, and Jordans (2000b) intentional influences model, and use these feature
sets in a machine learning experiment to automatically learn a model of content
selection for object descriptions. Since Dale and Reiters model requires a
representation of discourse structure, the corpus annotations are used to
derive a representation based on Grosz and Sidners (1986) theory of the
intentional structure of discourse, as well as two very simple representations
of discourse structure based purely on recency. We then apply the
rule-induction program RIPPER to train and test the content selection component
of an object description generator on a set of 393 object descriptions from the
corpus. To our knowledge, this is the first reported experiment of a trainable
content selection component for object description generation in dialogue.
Three separate content selection models that are based on the three theoretical
models, all independently achieve accuracies significantly above the majority
class baseline (17%) on unseen test data, with the intentional influences model
(42.4%) performing significantly better than either the incremental model
(30.4%) or the conceptual pact model (28.9%). But the best performing models
combine all the feature sets, achieving accuracies near 60%. Surprisingly, a
simple recency-based representation of discourse structure does as well as one
based on intentional structure. To our knowledge, this is also the first
empirical comparison of a representation of Grosz and Sidners model of
discourse structure with a simpler model for any generation task.

本文研究聊天机器人生成任务型对话中的对象描述内容选取这一子问题，使用基于不同模型的特征集进行机器学习并训练出选择对象描述内容的组件，其中包括递增模型、概念协定模型和意图影响模型。同时，本文还与简单的 recentity 模型进行了对比实验，发现意图影响模型表现最佳。