Data poisoning considers an adversary that distorts the training set of
machine learning algorithms for malicious purposes. In this work, we bring to
light one conjecture regarding the fundamentals of data poisoning, which we
call the Lethal Dose Conjecture. The conjecture states: If $n$ clean training
samples are needed for accurate predictions, then in a size-$N$ training set,
only $\Theta(N/n)$ poisoned samples can be tolerated while ensuring accuracy.
Theoretically, we verify this conjecture in multiple cases. We also offer a
more general perspective of this conjecture through distribution
discrimination. Deep Partition Aggregation (DPA) and its extension, Finite
Aggregation (FA) are recent approaches for provable defenses against data
poisoning, where they predict through the majority vote of many base models
trained from different subsets of training set using a given learner. The
conjecture implies that both DPA and FA are (asymptotically) optimal -- if we
have the most data-efficient learner, they can turn it into one of the most
robust defenses against data poisoning. This outlines a practical approach to
developing stronger defenses against poisoning via finding data-efficient
learners. Empirically, as a proof of concept, we show that by simply using
different data augmentations for base learners, we can respectively double and
triple the certified robustness of DPA on CIFAR-10 and GTSRB without
sacrificing accuracy.

此论文提出了数据污染及其危害的基本概念，阐述了所谓的致命剂量猜想，为开发更强大的数据污染防御策略提供了实用方法，并通过数据扩充来验证 DPA 的有效性。

关于数据投毒的致死剂量猜想

Lethal Dose Conjecture on Data Poisoning

Pre-training video transformers on extra large-scale datasets is generally
required to achieve premier performance on relatively small datasets. In this
paper, we show that video masked autoencoders (VideoMAE) are data-efficient
learners for self-supervised video pre-training (SSVP). We are inspired by the
recent ImageMAE and propose customized video tube masking with an extremely
high ratio. This simple design makes video reconstruction a more challenging
self-supervision task, thus encouraging extracting more effective video
representations during this pre-training process. We obtain three important
findings on SSVP: (1) An extremely high proportion of masking ratio (i.e., 90%
to 95%) still yields favorable performance of VideoMAE. The temporally
redundant video content enables a higher masking ratio than that of images. (2)
VideoMAE achieves impressive results on very small datasets (i.e., around 3k-4k
videos) without using any extra data. (3) VideoMAE shows that data quality is
more important than data quantity for SSVP. Domain shift between pre-training
and target datasets is an important issue. Notably, our VideoMAE with the
vanilla ViT can achieve 87.4% on Kinetics-400, 75.4% on Something-Something V2,
91.3% on UCF101, and 62.6% on HMDB51, without using any extra data. Code is
available at this https URL

本文研究了使用 VideoMAE 进行自主监督视频预训练 (SSVP) 的数据高效性问题，并通过适当的视频屏蔽达到了良好的表现，进而证明数据质量对 SSVP 更加重要。