Recently, binary representation has been proposed as a novel representation
that lies between continuous and discrete representations. It exhibits
considerable information-preserving capability when being used to replace
continuous input vectors. In this paper, we investigate the feasibility of
further introducing it to the output side, aiming to allow models to output
binary labels instead. To preserve the structural information on the output
side along with label information, we extend the previous contrastive hashing
method as structured contrastive hashing. More specifically, we upgrade CKY
from label-level to bit-level, define a new similarity function with span
marginal probabilities, and introduce a novel contrastive loss function with a
carefully designed instance selection strategy. Our model achieves competitive
performance on various structured prediction tasks, and demonstrates that
binary representation can be considered a novel representation that further
bridges the gap between the continuous nature of deep learning and the discrete
intrinsic property of natural languages.

本文研究了将二元表示引入输出端，以允许模型输出二元标签，通过改进 CKY 算法、定义新的相似性函数和引入新的对比损失函数，我们的模型在各种结构化预测任务中取得了有竞争力的性能，并证明了二元表示可以进一步缩小深度学习连续性和自然语言离散性之间的差距。