Semantic communications (SC) have been expected to be a new paradigm shifting
to catalyze the next generation communication, whose main concerns shift from
accurate bit transmission to effective semantic information exchange in
communications. However, the previous and widely-used metrics for images are
not applicable to evaluate the image semantic similarity in SC. Classical
metrics to measure the similarity between two images usually rely on the pixel
level or the structural level, such as the PSNR and the MS-SSIM.
Straightforwardly using some tailored metrics based on deep-learning methods in
CV community, such as the LPIPS, is infeasible for SC. To tackle this, inspired
by BERTScore in NLP community, we propose a novel metric for evaluating image
semantic similarity, named Vision Transformer Score (ViTScore). We prove
theoretically that ViTScore has 3 important properties, including symmetry,
boundedness, and normalization, which make ViTScore convenient and intuitive
for image measurement. To evaluate the performance of ViTScore, we compare
ViTScore with 3 typical metrics (PSNR, MS-SSIM, and LPIPS) through 5 classes of
experiments. Experimental results demonstrate that ViTScore can better evaluate
the image semantic similarity than the other 3 typical metrics, which indicates
that ViTScore is an effective performance metric when deployed in SC scenarios.

语义通信（SC）是一个新的范式，其主要关注点从准确的比特传输转变为在通信中进行有效的语义信息交换；本研究提出了一种评估图像语义相似性的新指标 ViTScore，并通过与其他三种典型指标的比较，证明了 ViTScore 在 SC 场景中更好地评估图像的语义相似性，表明它是一种有效的性能指标。