Sociodemographic factors (e.g., gender or age) shape our language. Previous
work showed that incorporating specific sociodemographic factors can
consistently improve performance for various NLP tasks in traditional NLP
models. We investigate whether these previous findings still hold with
state-of-the-art pretrained Transformers. We use three common specialization
methods proven effective for incorporating external knowledge into pretrained
Transformers (e.g., domain-specific or geographic knowledge). We adapt the
language representations for the sociodemographic dimensions of gender and age,
using continuous language modeling and dynamic multi-task learning for
adaptation, where we couple language modeling with the prediction of a
sociodemographic class. Our results when employing a multilingual model show
substantial performance gains across four languages (English, German, French,
and Danish). These findings are in line with the results of previous work and
hold promise for successful sociodemographic specialization. However,
controlling for confounding factors like domain and language shows that, while
sociodemographic adaptation does improve downstream performance, the gains do
not always solely stem from sociodemographic knowledge. Our results indicate
that sociodemographic specialization, while very important, is still an
unresolved problem in NLP.

本文旨在探究在新型预训练变压器模型下是否也存在特定群体的社会人口因素能够大幅提高自然语言处理任务性能的情形，研究使用领域特定知识等有效的特化方法对性别和年龄这两个社会人口维度进行了适应，通过语言建模和动态多任务学习的组合，将语言表示出社会人口类别的预测，并在英语、德语、法语和丹麦语等四种语言中得到明显的提升，但是，实验结果表明，需要解决领域和语言等诸多影响因素所带来的困扰，才能权衡全面考虑社会人口因素对 NLP 模型的影响。

Transformer 模型在社会人口调整中的局限性

On the Limitations of Sociodemographic Adaptation with Transformers

Past research has shown the benefits of food journaling in promoting mindful
eating and healthier food choices. However, the links between journaling and
healthy eating have not been thoroughly examined. Beyond caloric restriction,
do journalers consistently and sufficiently consume healthful diets? How
different are their eating habits compared to those of average consumers who
tend to be less conscious about health? In this study, we analyze the healthy
eating behaviors of active food journalers using data from MyFitnessPal.
Surprisingly, our findings show that food journalers do not eat as healthily as
they should despite their proclivity to health eating and their food choices
resemble those of the general populace. Furthermore, we find that the
journaling duration is only a marginal determinant of healthy eating outcomes
and sociodemographic factors, such as gender and regions of residence, are much
more predictive of healthy food choices.

研究了食品日志记录者的饮食行为，发现他们的饮食习惯与普通人相似，其健康饮食的影响因素与社会人口学因素更相关，而食品日志纪录的时间对健康的饮食选择影响较小。