Honey bees pollinate about one-third of the world's food supply, but bee
colonies have alarmingly declined by nearly 40% over the past decade due to
several factors, including pesticides and pests. Traditional methods for
monitoring beehives, such as human inspection, are subjective, disruptive, and
time-consuming. To overcome these limitations, artificial intelligence has been
used to assess beehive health. However, previous studies have lacked an
end-to-end solution and primarily relied on data from a single source, either
bee images or sounds. This study introduces a comprehensive system consisting
of bee object detection and health evaluation. Additionally, it utilized a
combination of visual and audio signals to analyze bee behaviors. An
Attention-based Multimodal Neural Network (AMNN) was developed to adaptively
focus on key features from each type of signal for accurate bee health
assessment. The AMNN achieved an overall accuracy of 92.61%, surpassing eight
existing single-signal Convolutional Neural Networks and Recurrent Neural
Networks. It outperformed the best image-based model by 32.51% and the top
sound-based model by 13.98% while maintaining efficient processing times.
Furthermore, it improved prediction robustness, attaining an F1-score higher
than 90% across all four evaluated health conditions. The study also shows that
audio signals are more reliable than images for assessing bee health. By
seamlessly integrating AMNN with image and sound data in a comprehensive bee
health monitoring system, this approach provides a more efficient and
non-invasive solution for the early detection of bee diseases and the
preservation of bee colonies.

蜜蜂在全球食品供应中起着大约三分之一的授粉作用，但由于多种因素，包括杀虫剂和害虫，蜜蜂群数量在过去十年中惊人地下降了近 40%。本研究通过引入综合系统，包括蜜蜂目标检测和健康评估，综合利用视觉和音频信号来分析蜜蜂行为，并开发了一种自适应关注多模态神经网络（AMNN）来准确评估蜜蜂健康状况。AMNN 在保持高效处理时间的同时，取得了 92.61% 的整体准确率，超过了八个现有的单一信号卷积神经网络和递归神经网络。此外，它提高了预测鲁棒性，在所有四种评估的健康状况中，F1 得分均超过 90%。此研究还表明相比图像，音频信号更可靠地评估蜜蜂健康状况。通过将 AMNN 与图像和声音数据无缝集成于综合的蜜蜂健康监测系统中，这种方法为早期检测蜜蜂疾病和保护蜜蜂群提供了更高效、无创的解决方案。