Smart Digital twins (SDTs) are being increasingly used to virtually replicate
and predict the behaviors of complex physical systems through continual data
assimilation enabling the optimization of the performance of these systems by
controlling the actions of systems. Recently, deep learning (DL) models have
significantly enhanced the capabilities of SDTs, particularly for tasks such as
predictive maintenance, anomaly detection, and optimization. In many domains,
including medicine, engineering, and education, SDTs use image data
(image-based SDTs) to observe and learn system behaviors and control their
behaviors. This paper focuses on various approaches and associated challenges
in developing image-based SDTs by continually assimilating image data from
physical systems. The paper also discusses the challenges involved in designing
and implementing DL models for SDTs, including data acquisition, processing,
and interpretation. In addition, insights into the future directions and
opportunities for developing new image-based DL approaches to develop robust
SDTs are provided. This includes the potential for using generative models for
data augmentation, developing multi-modal DL models, and exploring the
integration of DL with other technologies, including 5G, edge computing, and
IoT. In this paper, we describe the image-based SDTs, which enable broader
adoption of the digital twin DT paradigms across a broad spectrum of areas and
the development of new methods to improve the abilities of SDTs in replicating,
predicting, and optimizing the behavior of complex systems.

通过持续数据同化，智能数字孪生系统（SDTs）在多个领域中使用图像数据观察和学习系统行为，并控制其行为，从而提高复杂系统的性能优化。本文聚焦于不断同化物理系统中的图像数据来发展基于图像的 SDTs 的各种方法及相关挑战，并讨论了设计和实现 DL 模型用于 SDTs 所面临的挑战，包括数据采集、处理和解释等方面。此外，还提供了关于发展新的基于图像的 DL 方法以构建稳健 SDTs 的未来方向和机会，包括利用生成模型进行数据增强，开发多模态 DL 模型，以及探索与 5G、边缘计算和物联网等其他技术的融合。本文描述了基于图像的 SDTs，使得数字孪生系统（DT）的范式能够在广泛的领域得到更广泛的应用，并开发新的方法以提高 SDTs 在复制、预测和优化复杂系统行为方面的能力。