Paralinguistic traits like cognitive load and emotion are increasingly
recognized as pivotal areas in speech recognition research, often examined
through specialized datasets like CLSE and IEMOCAP. However, the integrity of
these datasets is seldom scrutinized for text-dependency. This paper critically
evaluates the prevalent assumption that machine learning models trained on such
datasets genuinely learn to identify paralinguistic traits, rather than merely
capturing lexical features. By examining the lexical overlap in these datasets
and testing the performance of machine learning models, we expose significant
text-dependency in trait-labeling. Our results suggest that some machine
learning models, especially large pre-trained models like HuBERT, might
inadvertently focus on lexical characteristics rather than the intended
paralinguistic features. The study serves as a call to action for the research
community to reevaluate the reliability of existing datasets and methodologies,
ensuring that machine learning models genuinely learn what they are designed to
recognize.

通过检查专门数据集中的词汇重叠和机器学习模型的性能，我们揭示了特质标签中的显著文本依赖性，结果表明一些机器学习模型可能过多关注词汇特征而不是目标的语音特征，研究呼吁研究界重新评估现有数据集和方法论的可靠性，确保机器学习模型真正学到其设计的内容。