Real-time Digital Twinning of physical world scenes onto the Metaverse is
necessary for a myriad of applications such as augmented-reality (AR) assisted
driving. In AR assisted driving, physical environment scenes are first captured
by Internet of Vehicles (IoVs) and are uploaded to the Metaverse. A central
Metaverse Map Service Provider (MMSP) will aggregate information from all IoVs
to develop a central Metaverse Map. Information from the Metaverse Map can then
be downloaded into individual IoVs on demand and be delivered as AR scenes to
the driver. However, the growing interest in developing AR assisted driving
applications which relies on digital twinning invites adversaries. These
adversaries may place physical adversarial patches on physical world objects
such as cars, signboards, or on roads, seeking to contort the virtual world
digital twin. Hence, there is a need to detect these physical world adversarial
patches. Nevertheless, as real-time, accurate detection of adversarial patches
is compute-intensive, these physical world scenes have to be offloaded to the
Metaverse Map Base Stations (MMBS) for computation. Hence in our work, we
considered an environment with moving Internet of Vehicles (IoV), uploading
real-time physical world scenes to the MMBSs. We formulated a realistic joint
variable optimization problem where the MMSPs' objective is to maximize
adversarial patch detection mean average precision (mAP), while minimizing the
computed AR scene up-link transmission latency and IoVs' up-link transmission
idle count, through optimizing the IoV-MMBS allocation and IoV up-link scene
resolution selection. We proposed a Heterogeneous Action Proximal Policy
Optimization (HAPPO) (discrete-continuous) algorithm to tackle the proposed
problem. Extensive experiments shows HAPPO outperforms baseline models when
compared against key metrics.

本研究旨在解决互联网车联网拍摄实时场景并将其投入虚拟现实的实时数字双生技术中，从而应用于增强现实辅助驾驶；同时，针对数字双生技术招致了对手攻击的问题，提出了一种基于异构行动近端策略优化（HAPPO）算法的解决方法。实验结果表明，HAPPO 在关键指标上优于基准模型。