We introduce a recurrent neural network language model (RNN-LM) with long
short-term memory (LSTM) units that utilizes both character-level and
word-level inputs. Our model has a gate that adaptively finds the optimal
mixture of the character-level and word-level inputs. The gate creates the
final vector representation of a word by combining two distinct representations
of the word. The character-level inputs are converted into vector
representations of words using a bidirectional LSTM. The word-level inputs are
projected into another high-dimensional space by a word lookup table. The final
vector representations of words are used in the LSTM language model which
predicts the next word given all the preceding words. Our model with the gating
mechanism effectively utilizes the character-level inputs for rare and
out-of-vocabulary words and outperforms word-level language models on several
English corpora.

本研究提出一种基于长短时记忆（LSTM）单元的循环神经网络语言模型（RNN-LM），有效地利用字符级和单词级输入，在预测英语文本中的下一个单词方面表现优异。

门控字、词循环语言模型

Gated Word-Character Recurrent Language Model

Document classification tasks were primarily tackled at word level. Recent
research that works with character-level inputs shows several benefits over
word-level approaches such as natural incorporation of morphemes and better
handling of rare words. We propose a neural network architecture that utilizes
both convolution and recurrent layers to efficiently encode character inputs.
We validate the proposed model on eight large scale document classification
tasks and compare with character-level convolution-only models. It achieves
comparable performances with much less parameters.

本研究提出了一种利用卷积和循环层来高效编码字符输入的神经网络架构，用于文档分类任务，与仅具有卷积层的字符级模型相比，具有相当的性能而参数更少。

通过结合卷积和循环层实现高效的字符级文档分类

Efficient Character-level Document Classification by Combining  Convolution and Recurrent Layers

We describe a simple neural language model that relies only on
character-level inputs. Predictions are still made at the word-level. Our model
employs a convolutional neural network (CNN) and a highway network over
characters, whose output is given to a long short-term memory (LSTM) recurrent
neural network language model (RNN-LM). On the English Penn Treebank the model
is on par with the existing state-of-the-art despite having 60% fewer
parameters. On languages with rich morphology (Arabic, Czech, French, German,
Spanish, Russian), the model outperforms word-level/morpheme-level LSTM
baselines, again with fewer parameters. The results suggest that on many
languages, character inputs are sufficient for language modeling. Analysis of
word representations obtained from the character composition part of the model
reveals that the model is able to encode, from characters only, both semantic
and orthographic information.

本研究描述了一种仅依赖字符级输入的简单神经语言模型，该模型利用卷积神经网络和字符级公路网络作为输入，并将结果给予一种长短时记忆递归神经网络语言模型作为输出，该模型可以用较少的参数占据现有最先进的地位。此外，该模型表现出色的证明了许多语言所需的字符输入已经足够进行语言建模，且可以从字符组成的部分编码的单词表示中得到语义和字形信息。