Keeping the performance of language technologies optimal as time passes is of
great practical interest. We study temporal effects on model performance on
downstream language tasks, establishing a nuanced terminology for such
discussion and identifying factors essential to conduct a robust study. We
present experiments for several tasks in English where the label correctness is
not dependent on time and demonstrate the importance of distinguishing between
temporal model deterioration and temporal domain adaptation for systems using
pre-trained representations. We find that depending on the task, temporal model
deterioration is not necessarily a concern. Temporal domain adaptation however
is beneficial in all cases, with better performance for a given time period
possible when the system is trained on temporally more recent data. Therefore,
we also examine the efficacy of two approaches for temporal domain adaptation
without human annotations on new data. Self-labeling shows consistent
improvement and notably, for named entity recognition, leads to better temporal
adaptation than even human annotations.

研究语言科技的效能随时间的变化是非常实用的。本文研究了模型性能在下游语言任务中的时间效应，建立了相应术语，识别了进行强有力研究的关键因素，以英语为例，展示了任务的实验，发现依赖于时间的标签正确性并不会降低模型性能，但考虑时域的适应是有益的。自标记表现最佳。