Machine learning applications on extremely low-power devices, commonly
referred to as tiny machine learning (TinyML), promises a smarter and more
connected world. However, the advancement of current TinyML research is
hindered by the limited size and quality of pertinent datasets. To address this
challenge, we introduce Wake Vision, a large-scale, diverse dataset tailored
for person detection -- the canonical task for TinyML visual sensing. Wake
Vision comprises over 6 million images, which is a hundredfold increase
compared to the previous standard, and has undergone thorough quality
filtering. Using Wake Vision for training results in a 2.41\% increase in
accuracy compared to the established benchmark. Alongside the dataset, we
provide a collection of five detailed benchmark sets that assess model
performance on specific segments of the test data, such as varying lighting
conditions, distances from the camera, and demographic characteristics of
subjects. These novel fine-grained benchmarks facilitate the evaluation of
model quality in challenging real-world scenarios that are often ignored when
focusing solely on overall accuracy. Through an evaluation of a MobileNetV2
TinyML model on the benchmarks, we show that the input resolution plays a more
crucial role than the model width in detecting distant subjects and that the
impact of quantization on model robustness is minimal, thanks to the dataset
quality. These findings underscore the importance of a detailed evaluation to
identify essential factors for model development. The dataset, benchmark suite,
code, and models are publicly available under the CC-BY 4.0 license, enabling
their use for commercial use cases.

通过介绍 Wake Vision 数据集，我们针对 TinyML 视觉感知的人员检测任务，提供了一种大规模、多样化的数据集，并在测试数据的特定段上提供了五个详细的基准集，以评估模型在挑战性现实场景中的性能，从而使得使用 Wake Vision 进行训练相对于已建立的基准可以提高 2.41％的准确性。