Previous work optimizes traditional active learning (AL) processes with
incremental neural network architecture search (Active-iNAS) based on data
complexity change, which improves the accuracy and learning efficiency.
However, Active-iNAS trains several models and selects the model with the best
generalization performance for querying the subsequent samples after each
active learning cycle. The independent training processes lead to an
insufferable computational budget, which is significantly inefficient and
limits search flexibility and final performance. To address this issue, we
propose a novel active strategy with the method called structured variational
inference (SVI) or structured neural depth search (SNDS) whereby we could use
the gradient descent method in neural network depth search during AL processes.
At the same time, we theoretically demonstrate that the current VI-based
methods based on the mean-field assumption could lead to poor performance. We
apply our strategy using three querying techniques and three datasets and show
that our strategy outperforms current methods.

本研究提出了一种使用结构化变分推断或结构化神经深度搜索的新型主动学习策略，在神经网络深度搜索期间使用梯度下降方法，并理论证明了基于平均场假设的当前 VI 方法可能导致性能不佳。通过应用三种查询技术和三个数据集，表明我们的策略优于当前方法。

带有结构化神经深度搜索的深度主动学习

Deep Active Learning with Structured Neural Depth Search

Recent developments in support for constructing knowledge graphs have led to
a rapid rise in their creation both on the Web and within organisations. Added
to existing sources of data, including relational databases, APIs, etc., there
is a strong demand for techniques to query these diverse sources of knowledge.
While formal query languages, such as SPARQL, exist for querying some knowledge
graphs, users are required to know which knowledge graphs they need to query
and the unique resource identifiers of the resources they need. Although
alternative techniques in neural information retrieval embed the content of
knowledge graphs in vector spaces, they fail to provide the representation and
query expressivity needed (e.g. inability to handle non-trivial aggregation
functions such as regression). We believe that a lingua franca, i.e. a
formalism, that enables such representational flexibility will increase the
ability of intelligent automated agents to combine diverse data sources by
inference.
Our work proposes a flexible representation (alists) to support intelligent
federated querying of diverse knowledge sources. Our contribution includes (1)
a formalism that abstracts the representation of queries from the specific
query language of a knowledge graph; (2) a representation to dynamically curate
data and functions (operations) to perform non-trivial inference over diverse
knowledge sources; (3) a demonstration of the expressiveness of alists to
represent the diversity of representational formalisms, including SPARQL
queries, and more generally first-order logic expressions.

本研究提出了一种灵活的表征方式（alists），以支持对不同知识源的智能联合查询和推理，解决了其他信息检索技术中无法处理非平凡聚合功能的问题，从而实现了代理能够合并不同数据源的能力。

ALIST: 基于关联逻辑的推理、存储和传输。用于网络推理的通用语言

ALIST: Associative Logic for Inference, Storage and Transfer. A Lingua Franca for Inference on the Web

We consider active learning of deep neural networks. Most active learning
works in this context have focused on studying effective querying mechanisms
and assumed that an appropriate network architecture is a priori known for the
problem at hand. We challenge this assumption and propose a novel active
strategy whereby the learning algorithm searches for effective architectures on
the fly, while actively learning. We apply our strategy using three known
querying techniques (softmax response, MC-dropout, and coresets) and show that
the proposed approach overwhelmingly outperforms active learning using fixed
architectures.

本文研究了主动学习深度神经网络，并提出了一种新的主动策略，该策略能够在学习过程中搜索有效的架构，同时使用三种已知的查询技术（softmax response，MC-dropout 和 coresets）进行应用，结果表明该方法在固定架构的主动学习方面表现出显著的优势。