Autocomplete is a task where the user inputs a piece of text, termed prompt,
which is conditioned by the model to generate semantically coherent
continuation. Existing works for this task have primarily focused on datasets
(e.g., email, chat) with high frequency user prompt patterns (or focused
prompts) where word-based language models have been quite effective. In this
work, we study the more challenging setting consisting of low frequency user
prompt patterns (or broad prompts, e.g., prompt about 93rd academy awards) and
demonstrate the effectiveness of character-based language models. We study this
problem under memory-constrained settings (e.g., edge devices and smartphones),
where character-based representation is effective in reducing the overall model
size (in terms of parameters). We use WikiText-103 benchmark to simulate broad
prompts and demonstrate that character models rival word models in exact match
accuracy for the autocomplete task, when controlled for the model size. For
instance, we show that a 20M parameter character model performs similar to an
80M parameter word model in the vanilla setting. We further propose novel
methods to improve character models by incorporating inductive bias in the form
of compositional information and representation transfer from large word
models.

本文研究的是低频用户提示模式或广泛提示的自动完成任务以及在受限内存下使用基于字符的语言模型降低整体模型大小的效果。研究证明，当控制模型大小时，字符模型的自动完成任务的精确匹配准确性与字模型相当。作者进一步尝试在字符模型中融入大型字模型的组成信息和表现转移的归纳偏差。

受内存限制的自动完成中，小字符模型能与大词模型匹配

Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints

We propose an attentive neural network for the task of named entity
recognition in Vietnamese. The proposed attentive neural model makes use of
character-based language models and word embeddings to encode words as vector
representations. A neural network architecture of encoder, attention, and
decoder layers is then utilized to encode knowledge of input sentences and to
label entity tags. The experimental results show that the proposed attentive
neural network achieves the state-of-the-art results on the benchmark named
entity recognition datasets in Vietnamese in comparison to both hand-crafted
features based models and neural models.

提出了一种用于越南语命名实体识别的注意力神经网络，其利用基于字符的语言模型和词嵌入来编码单词向量表示，并利用编码器、注意力和解码器层的神经网络架构来编码输入句子的知识和标记实体标签。实验结果表明，与手工制作特征模型和神经模型相比，所提出的注意力神经网络在越南的基准命名实体识别数据集上实现了最先进的结果。