We propose a communication-efficient collaborative inference framework in the
domain of edge inference, focusing on the efficient use of vision transformer
(ViTs) models. The partitioning strategy of conventional collaborative
inference fails to reduce communication cost because of the inherent
architecture of ViTs maintaining consistent layer dimensions across the entire
transformer encoder. Therefore, instead of employing the partitioning strategy,
our framework utilizes a lightweight ViT model on the edge device, with the
server deploying a complicated ViT model. To enhance communication efficiency
and achieve the classification accuracy of the server model, we propose two
strategies: 1) attention-aware patch selection and 2) entropy-aware image
transmission. Attention-aware patch selection leverages the attention scores
generated by the edge device's transformer encoder to identify and select the
image patches critical for classification. This strategy enables the edge
device to transmit only the essential patches to the server, significantly
improving communication efficiency. Entropy-aware image transmission uses
min-entropy as a metric to accurately determine whether to depend on the
lightweight model on the edge device or to request the inference from the
server model. In our framework, the lightweight ViT model on the edge device
acts as a semantic encoder, efficiently identifying and selecting the crucial
image information required for the classification task. Our experiments
demonstrate that the proposed collaborative inference framework can reduce
communication overhead by 68% with only a minimal loss in accuracy compared to
the server model.

提出一种在边缘推理领域内，通过有效利用视觉 Transformer（ViTs）模型的通信高效的协同推理框架。该框架运用了轻量级的 ViT 模型在边缘设备上，服务器采用复杂的 ViT 模型，以提高通信效率并实现与服务器模型相当的分类准确性，通过关注度感知的补丁选择和熵感知的图像传输策略实现。实验证明，该协同推理框架可以将通信开销减少 68%，同时仅有最小的准确性损失。