Review comments play an important role in the evolution of documents. For a
large document, the number of review comments may become large, making it
difficult for the authors to quickly grasp what the comments are about. It is
important to identify the nature of the comments to identify which comments
require some action on the part of document authors, along with identifying the
types of these comments. In this paper, we introduce an annotated review
comment dataset ReAct. The review comments are sourced from OpenReview site. We
crowd-source annotations for these reviews for actionability and type of
comments. We analyze the properties of the dataset and validate the quality of
annotations. We release the dataset (this https URL) to the
research community as a major contribution. We also benchmark our data with
standard baselines for classification tasks and analyze their performance.

本研究介绍了一个从 OpenReview 网站中获取注释评论的数据集 ReAct，对这些评论进行了行动性和类型的众包注释。该数据集可用于评论分类等任务评估和性能分析。

ReAct: 用于行动性 (以及更多) 的评论回顾数据集

ReAct: A Review Comment Dataset for Actionability (and more)

In this work, we apply word embeddings and neural networks with Long
Short-Term Memory (LSTM) to text classification problems, where the
classification criteria are decided by the context of the application. We
examine two applications in particular. The first is that of Actionability,
where we build models to classify social media messages from customers of
service providers as Actionable or Non-Actionable. We build models for over 30
different languages for actionability, and most of the models achieve accuracy
around 85%, with some reaching over 90% accuracy. We also show that using LSTM
neural networks with word embeddings vastly outperform traditional techniques.
Second, we explore classification of messages with respect to political
leaning, where social media messages are classified as Democratic or
Republican. The model is able to classify messages with a high accuracy of
87.57%. As part of our experiments, we vary different hyperparameters of the
neural networks, and report the effect of such variation on the accuracy. These
actionability models have been deployed to production and help company agents
provide customer support by prioritizing which messages to respond to. The
model for political leaning has been opened and made available for wider use.

本文利用词嵌入和具有长短时记忆的神经网络进行文本分类，其中分类标准由应用背景决定。我们在行动能力和政治倾向两个应用领域进行了实验，证明了使用 LSTM 神经网络和词嵌入技术远优于传统技术。其中，我们为 30 多种语言构建了行动能力模型，并展示了这些模型在精度方面能达到 85% 左右。我们还展示了如何利用这些模型，以帮助公司代理人员优先处理需要处理的客户信息。