People convey their intention and attitude through linguistic styles of the
text that they write. In this study, we investigate lexicon usages across
styles throughout two lenses: human perception and machine word importance,
since words differ in the strength of the stylistic cues that they provide. To
collect labels of human perception, we curate a new dataset, Hummingbird, on
top of benchmarking style datasets. We have crowd workers highlight the
representative words in the text that makes them think the text has the
following styles: politeness, sentiment, offensiveness, and five emotion types.
We then compare these human word labels with word importance derived from a
popular fine-tuned style classifier like BERT. Our results show that the BERT
often finds content words not relevant to the target style as important words
used in style prediction, but humans do not perceive the same way even though
for some styles (e.g., positive sentiment and joy) human- and
machine-identified words share significant overlap for some styles.

本研究通过探究人类感知和机器词汇重要性这两个视角，研究了文本的语言风格对词汇用法的影响，通过收集人类感知数据 Hummingbird 并参考目前常用的 BERT 语言风格分类器，比较了两者对于词汇标签的不同理解。研究表明，对于某些风格，如积极情感和愉悦，人类和机器的识别结果存在显著的重叠，但机器标记某些内容词非风格相关，而人们不能通过这些词汇准确感知对应的风格。