Word embeddings are commonly used as a starting point in many NLP models to
achieve state-of-the-art performances. However, with a large vocabulary and
many dimensions, these floating-point representations are expensive both in
terms of memory and calculations which makes them unsuitable for use on
low-resource devices. The method proposed in this paper transforms real-valued
embeddings into binary embeddings while preserving semantic information,
requiring only 128 or 256 bits for each vector. This leads to a small memory
footprint and fast vector operations. The model is based on an autoencoder
architecture, which also allows to reconstruct original vectors from the binary
ones. Experimental results on semantic similarity, text classification and
sentiment analysis tasks show that the binarization of word embeddings only
leads to a loss of ~2% in accuracy while vector size is reduced by 97%.
Furthermore, a top-k benchmark demonstrates that using these binary vectors is
30 times faster than using real-valued vectors.

本文提出一种基于自编码器的方法将实值的词向量转化为二进制向量，从而在减小空间占用的同时只损失了 2% 的精度。实验结果表明，使用这些二进制向量比使用实值向量快 30 倍。