Vision Transformers (ViTs) have excelled in computer vision tasks but are
memory-consuming and computation-intensive, challenging their deployment on
resource-constrained devices. To tackle this limitation, prior works have
explored ViT-tailored quantization algorithms but retained floating-point
scaling factors, which yield non-negligible re-quantization overhead, limiting
ViTs' hardware efficiency and motivating more hardware-friendly solutions. To
this end, we propose \emph{P$^2$-ViT}, the first \underline{P}ower-of-Two (PoT)
\underline{p}ost-training quantization and acceleration framework to accelerate
fully quantized ViTs. Specifically, {as for quantization,} we explore a
dedicated quantization scheme to effectively quantize ViTs with PoT scaling
factors, thus minimizing the re-quantization overhead. Furthermore, we propose
coarse-to-fine automatic mixed-precision quantization to enable better
accuracy-efficiency trade-offs. {In terms of hardware,} we develop {a dedicated
chunk-based accelerator} featuring multiple tailored sub-processors to
individually handle ViTs' different types of operations, alleviating
reconfigurable overhead. Additionally, we design {a tailored row-stationary
dataflow} to seize the pipeline processing opportunity introduced by our PoT
scaling factors, thereby enhancing throughput. Extensive experiments
consistently validate P$^2$-ViT's effectiveness. {Particularly, we offer
comparable or even superior quantization performance with PoT scaling factors
when compared to the counterpart with floating-point scaling factors. Besides,
we achieve up to $\mathbf{10.1\times}$ speedup and $\mathbf{36.8\times}$ energy
saving over GPU's Turing Tensor Cores, and up to $\mathbf{1.84\times}$ higher
computation utilization efficiency against SOTA quantization-based ViT
accelerators. Codes are available at
https://github.com/shihuihong214/P2-ViT.

Vision Transformers（ViTs）在计算机视觉任务中表现卓越，但它们在资源受限设备上的部署受到内存消耗和计算密集性的挑战。为了解决这个限制，我们提出了第一个适用于 P2-ViT 的后训练量化和加速框架，通过专门的量化方案有效量化 ViTs，并设计硬件加速器和数据流来提高效率。