This paper presents a novel multi modal deep learning framework for enhanced agricultural pest detection, combining tiny-BERT's natural language processing with R-CNN and ResNet-18's image processing. Addressing limitations of traditional CNN-based visual methods, this approach integrates textual context for more accurate pest identification. The R-CNN and ResNet-18 integration tackles deep CNN issues like vanishing gradients, while tiny-BERT ensures computational efficiency. Employing ensemble learning with linear regression and random forest models, the framework demonstrates superior discriminate ability, as shown in ROC and AUC analyses. This multi modal approach, blending text and image data, significantly boosts pest detection in agriculture. The study highlights the potential of multi modal deep learning in complex real-world scenarios, suggesting future expansions in diversity of datasets, advanced data augmentation, and cross-modal attention mechanisms to enhance model performance.

本文提出了一种新颖的多模态深度学习框架，结合了tiny-BERT的自然语言处理和R-CNN以及ResNet-18的图像处理，以增强农业害虫检测。通过集成文本上下文进行更精确的害虫识别，该方法解决了传统基于CNN的视觉方法的局限性。R-CNN和ResNet-18的整合解决了深度CNN的问题，如梯度消失，而tiny-BERT确保了计算效率。通过线性回归和随机森林模型的集成学习，该框架展示了出色的判别能力，如ROC和AUC分析所示。将文本和图像数据融合的这种多模态方法显著提高了农业害虫检测的效果。研究突出了多模态深度学习在复杂的真实场景中的潜力，并建议在多样化的数据集、高级数据增强和跨模态关注机制方面扩展以提高模型性能。

高级虫害检测与分类的多模态方法