Deep neural networks have exhibited substantial success in the field of
Natural Language Processing (NLP) and ensuring their safety and reliability is
crucial: there are safety critical contexts where such models must be robust to
variability or attack, and give guarantees over their output. Unlike Computer
Vision, NLP lacks a unified verification methodology and, despite recent
advancements in literature, they are often light on the pragmatical issues of
NLP verification. In this paper, we make an attempt to distil and evaluate
general components of an NLP verification pipeline, that emerges from the
progress in the field to date. Our contributions are two-fold. Firstly, we give
a general characterisation of verifiable subspaces that result from embedding
sentences into continuous spaces. We identify, and give an effective method to
deal with, the technical challenge of semantic generalisability of verified
subspaces; and propose it as a standard metric in the NLP verification
pipelines (alongside with the standard metrics of model accuracy and model
verifiability). Secondly, we propose a general methodology to analyse the
effect of the embedding gap, a problem that refers to the discrepancy between
verification of geometric subpspaces on the one hand, and semantic meaning of
sentences which the geometric subspaces are supposed to represent, on the other
hand. In extreme cases, poor choices in embedding of sentences may invalidate
verification results. We propose a number of practical NLP methods that can
help to identify the effects of the embedding gap; and in particular we propose
the metric of falsifiability of semantic subpspaces as another fundamental
metric to be reported as part of the NLP verification pipeline. We believe that
together these general principles pave the way towards a more consolidated and
effective development of this new domain.

通过总结现有领域进展，本文试图提炼和评估自然语言处理（NLP）验证流程的一般组成部分并分别给出了两个贡献，首先是针对将句子嵌入连续空间引起的语义一般化技术挑战，提出了一个有效的解决方法作为 NLP 验证流程的标准度量标准之一；其次是针对几何子空间验证和其所代表的句子语义之间的差异提出了一般性的分析方法，并提出了众多实用的 NLP 方法来识别嵌入空间差异的影响，并倡导以语义子空间可伪造性作为 NLP 验证流程的另一个基本度量标准，我们相信这些通用原则共同为这一新领域的更加巩固和有效的发展铺平了道路。

NLP 验证：向着一种通用的用于证明鲁棒性的方法论

NLP Verification: Towards a General Methodology for Certifying  Robustness

This paper presents an approach for grounding phrases in images which jointly
learns multiple text-conditioned embeddings in a single end-to-end model. In
order to differentiate text phrases into semantically distinct subspaces, we
propose a concept weight branch that automatically assigns phrases to
embeddings, whereas prior works predefine such assignments. Our proposed
solution simplifies the representation requirements for individual embeddings
and allows the underrepresented concepts to take advantage of the shared
representations before feeding them into concept-specific layers. Comprehensive
experiments verify the effectiveness of our approach across three phrase
grounding datasets, Flickr30K Entities, ReferIt Game, and Visual Genome, where
we obtain a (resp.) 4%, 3%, and 4% improvement in grounding performance over a
strong region-phrase embedding baseline.

本文提出了一种基于图像的短语 grounding 方法，基于一个端到端模型的多重条件嵌入来实现。为了将文本短语划分为语义上的不同子空间，我们提出了一个概念权重分支，可以自动将短语分配到嵌入，而不是像传统方法一样预先定义这些分配。我们的方法简化了个体嵌入的表征需求，并允许未被充分表示的概念在输入到概念特定层之前充分利用共享表示。在三个短语 grounding 数据集上的综合实验验证了我们方法的有效性，从而获得了强大的区域 - 短语嵌入基线 4％，3％和 4％ 的性能改进。

条件图像 - 文本嵌入网络

Conditional Image-Text Embedding Networks

What makes images similar? To measure the similarity between images, they are
typically embedded in a feature-vector space, in which their distance preserve
the relative dissimilarity. However, when learning such similarity embeddings
the simplifying assumption is commonly made that images are only compared to
one unique measure of similarity. A main reason for this is that contradicting
notions of similarities cannot be captured in a single space. To address this
shortcoming, we propose Conditional Similarity Networks (CSNs) that learn
embeddings differentiated into semantically distinct subspaces that capture the
different notions of similarities. CSNs jointly learn a disentangled embedding
where features for different similarities are encoded in separate dimensions as
well as masks that select and reweight relevant dimensions to induce a subspace
that encodes a specific similarity notion. We show that our approach learns
interpretable image representations with visually relevant semantic subspaces.
Further, when evaluating on triplet questions from multiple similarity notions
our model even outperforms the accuracy obtained by training individual
specialized networks for each notion separately.

本篇论文提出了一种称为 “条件相似性网络” 的方法，用于学习分离式嵌入，不同的相似性概念被编码在不同的语义子空间，并用于解决多个不同相似性概念的问题。