Neural Architecture Search (NAS) has shown promising capability in learning
text representation. However, existing text-based NAS neither performs a
learnable fusion of neural operations to optimize the architecture, nor encodes
the latent hierarchical categorization behind text input. This paper presents a
novel NAS method, Discretized Differentiable Neural Architecture Search
(DDNAS), for text representation learning and classification. With the
continuous relaxation of architecture representation, DDNAS can use gradient
descent to optimize the search. We also propose a novel discretization layer
via mutual information maximization, which is imposed on every search node to
model the latent hierarchical categorization in text representation. Extensive
experiments conducted on eight diverse real datasets exhibit that DDNAS can
consistently outperform the state-of-the-art NAS methods. While DDNAS relies on
only three basic operations, i.e., convolution, pooling, and none, to be the
candidates of NAS building blocks, its promising performance is noticeable and
extensible to obtain further improvement by adding more different operations.

本文提出了离散可微分神经架构搜索新方法（DDNAS）进行文本表示和分类的优化，并采用互信息最大化的离散化层来对文本表示中的潜在分层进行建模。实验证明，DDNAS 可显著优于现有的最先进的 NAS 方法。