New events emerge over time influencing the topics of rumors in social media.
Current rumor detection benchmarks use random splits as training, development
and test sets which typically results in topical overlaps. Consequently, models
trained on random splits may not perform well on rumor classification on
previously unseen topics due to the temporal concept drift. In this paper, we
provide a re-evaluation of classification models on four popular rumor
detection benchmarks considering chronological instead of random splits. Our
experimental results show that the use of random splits can significantly
overestimate predictive performance across all datasets and models. Therefore,
we suggest that rumor detection models should always be evaluated using
chronological splits for minimizing topical overlaps.

本研究通过对四种常用谣言检测基准数据集进行实验，发现使用时间序列而非随机分割数据集可以有效降低主题重叠部分，降低对模型预测性能高估的影响，建议谣言检测模型均应使用时间序列分割数据集以提高准确性。

重新审视谣言检测基准评估：基于时间序列的切分

It's about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits

Diarization partitions an audio stream into segments based on the voices of
the speakers. Real-time diarization systems that include an enrollment step
should limit enrollment training samples to reduce user interaction time.
Although training on a small number of samples yields poor performance, we show
that the accuracy can be improved dramatically using a chronological
self-training approach. We studied the tradeoff between training time and
classification performance and found that 1 second is sufficient to reach over
95% accuracy. We evaluated on 700 audio conversation files of about 10 minutes
each from 6 different languages and demonstrated average diarization error
rates as low as 10%.

该研究使用时间排序的自我训练方法，证明只使用 1 秒的样本训练可以将语音流分为不同的讲话者，并在多种语言中将错误率降低至 10% 以下。