We investigate how well CLIP understands texture in natural images described
by natural language. To this end, we analyze CLIP's ability to: (1) perform
zero-shot learning on various texture and material classification datasets; (2)
represent compositional properties of texture such as red dots or yellow
stripes on the Describable Texture in Detail(DTDD) dataset; and (3) aid
fine-grained categorization of birds in photographs described by color and
texture of their body parts.

本研究探讨了 CLIP 在自然语言描述的自然图像中对纹理的理解能力。我们分析了 CLIP 在各种纹理和材质分类数据集上的零样本学习表现，分析了它对 DTDD 数据集上红点或黄色条纹等纹理组成特性的表达能力，以及对通过描述鸟身体部位的颜色和纹理来进行细粒度分类的帮助。

CLIP 对纹理的理解能力如何？

How well does CLIP understand texture?

The behavior of self-driving cars must be compatible with an enormous set of
conflicting and ambiguous objectives, from law, from ethics, from the local
culture, and so on. This paper describes a new way to conveniently define the
desired behavior for autonomous agents, which we use on the self-driving cars
developed at nuTonomy. We define a "rulebook" as a pre-ordered set of "rules",
each akin to a violation metric on the possible outcomes ("realizations"). The
rules are partially ordered by priority. The semantics of a rulebook imposes a
pre-order on the set of realizations. We study the compositional properties of
the rulebooks, and we derive which operations we can allow on the rulebooks to
preserve previously-introduced constraints. While we demonstrate the
application of these techniques in the self-driving domain, the methods are
domain-independent.

该论文介绍了一种定义自主体所需行为的新方法 ——“规则书”，包括一系列 “规则”，每个规则类似于对可能结果（“实现”）的违规度量，并通过优先级部分排序。通过规则书的语义，我们可以对实现的集合施加预排序，研究了规则书的组合特性，并得出了如何在规则书上进行操作以保留先前引入的约束条件。尽管我们在自动驾驶领域展示了这些技术的应用，但这些方法是领域无关的。

使用规则书指定责任、伦理和文化感知行为

Liability, Ethics, and Culture-Aware Behavior Specification using  Rulebooks

Future predictions on sequence data (e.g., videos or audios) require the
algorithms to capture non-Markovian and compositional properties of high-level
semantics. Context-free grammars are natural choices to capture such
properties, but traditional grammar parsers (e.g., Earley parser) only take
symbolic sentences as inputs. In this paper, we generalize the Earley parser to
parse sequence data which is neither segmented nor labeled. This generalized
Earley parser integrates a grammar parser with a classifier to find the optimal
segmentation and labels, and makes top-down future predictions. Experiments
show that our method significantly outperforms other approaches for future
human activity prediction.

本研究旨在解决对未来序列数据的预测问题，提出将 Earley 解析器推广至未分段无标签序列数据，利用语言模型对最优的分段与标签进行预测，取得了显著的人类活动预测性能优于传统方法。