Reading dirty license plates accurately in moving vehicles is challenging for
automatic license plate recognition systems. Moreover, license plates are often
intentionally tampered with a malicious intent to avoid police apprehension.
Usually, such groups and individuals know how to fool the existing recognition
systems by making minor unnoticeable plate changes. Designing and developing
deep learning methods resilient to such real-world 'attack' practices remains
an active research problem. As a solution, this work develops a resilient
method to recognize license plate characters. Extracting 1057 character images
from 160 Nepalese vehicles, as the first step, we trained several standard deep
convolutional neural networks to obtain 99.5% character classification
accuracy. On adversarial images generated to simulate malicious tampering,
however, our model's accuracy dropped to 25%. Next, we enriched our dataset by
generating and adding geometrically masked images, retrained our models, and
investigated the models' predictions. The proposed approach of training with
generated adversarial images helped our adversarial attack-aware license plate
character recognition (AA-LPCR) model achieves an accuracy of 99.7%. This
near-perfect accuracy demonstrates that the proposed idea of random geometric
masking is highly effective for improving the accuracy of license plate
recognition models. Furthermore, by performing interpretability studies to
understand why our models work, we identify and highlight attack-prone regions
in the input character images. In sum, although Nepal's embossed license plate
detection systems are vulnerable to malicious attacks, our findings suggest
that these systems can be upgraded to close to 100% resilience.

通过使用深度学习方法，本研究通过对海量车牌字符图像的训练，提出一种具有抵抗故意篡改攻击的车牌字符识别方法。通过生成对抗性图像进行训练，模型在识别车牌字符方面达到了 99.7% 的准确率，并通过解释性研究找出了容易受到攻击的车牌字符图像区域，从而为改进车牌识别系统的鲁棒性提供了重要参考。