Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based
on propositional logic, which has demonstrated competitive performance in many
Natural Language Processing (NLP) tasks, including sentiment analysis, text
classification, and Word Sense Disambiguation. To obtain human-level
interpretability, legacy TM employs Boolean input features such as bag-of-words
(BOW). However, the BOW representation makes it difficult to use any
pre-trained information, for instance, word2vec and GloVe word representations.
This restriction has constrained the performance of TM compared to deep neural
networks (DNNs) in NLP. To reduce the performance gap, in this paper, we
propose a novel way of using pre-trained word representations for TM. The
approach significantly enhances the performance and interpretability of TM. We
achieve this by extracting semantically related words from pre-trained word
representations as input features to the TM. Our experiments show that the
accuracy of the proposed approach is significantly higher than the previous
BOW-based TM, reaching the level of DNN-based models.

本文提出了一种新的使用预训练词向量表示的 Tsetlin 机器方法，通过从预训练的词向量中提取语义相关的单词作为输入特征来增强 TM 的性能和可解释性。该方法的精度显著高于以前基于 BOW 的 TM，达到了 DNN-based 模型的水平。

利用预训练的词向量增强可解释的语义子句

Enhancing Interpretable Clauses Semantically using Pretrained Word  Representation

We demonstrate that replacing an LSTM encoder with a self-attentive
architecture can lead to improvements to a state-of-the-art discriminative
constituency parser. The use of attention makes explicit the manner in which
information is propagated between different locations in the sentence, which we
use to both analyze our model and propose potential improvements. For example,
we find that separating positional and content information in the encoder can
lead to improved parsing accuracy. Additionally, we evaluate different
approaches for lexical representation. Our parser achieves new state-of-the-art
results for single models trained on the Penn Treebank: 93.55 F1 without the
use of any external data, and 95.13 F1 when using pre-trained word
representations. Our parser also outperforms the previous best-published
accuracy figures on 8 of the 9 languages in the SPMRL dataset.

通过将 LSTM 编码器替换为自注意力机制，能够提高最先进的判别型成分解析器的性能，特别是当用预训练的字词表示时，而且这种方法在 SPMRL 数据集的大部分语言上的性能优于以前的最佳结果。

自注意力编码器进行的组成成分句法分析

Constituency Parsing with a Self-Attentive Encoder

Many Natural Language Processing applications nowadays rely on pre-trained
word representations estimated from large text corpora such as news
collections, Wikipedia and Web Crawl. In this paper, we show how to train
high-quality word vector representations by using a combination of known tricks
that are however rarely used together. The main result of our work is the new
set of publicly available pre-trained models that outperform the current state
of the art by a large margin on a number of tasks.

本文着重讨论了如何通过使用已知技巧来训练高质量的单词向量表示，以提高自然语言处理任务的表现，并提供了一组公开可用的预训练模型，能在多项任务上远远优于现有技术。