Diffusion models have achieved significant visual generation quality.
However, their significant computational and memory costs pose challenge for
their application on resource-constrained mobile devices or even desktop GPUs.
Recent few-step diffusion models reduces the inference time by reducing the
denoising steps. However, their memory consumptions are still excessive. The
Post Training Quantization (PTQ) replaces high bit-width FP representation with
low-bit integer values (INT4/8) , which is an effective and efficient technique
to reduce the memory cost. However, when applying to few-step diffusion models,
existing quantization methods face challenges in preserving both the image
quality and text alignment. To address this issue, we propose an
mixed-precision quantization framework - MixDQ. Firstly, We design specialized
BOS-aware quantization method for highly sensitive text embedding quantization.
Then, we conduct metric-decoupled sensitivity analysis to measure the
sensitivity of each layer. Finally, we develop an integer-programming-based
method to conduct bit-width allocation. While existing quantization methods
fall short at W8A8, MixDQ could achieve W8A8 without performance loss, and W4A8
with negligible visual degradation. Compared with FP16, we achieve 3-4x
reduction in model size and memory cost, and 1.45x latency speedup.

通过开发一种混合精度量化框架 MixDQ，我们针对强敏感度文本嵌入进行专门的 BOS 感知量化方法设计，通过度量解耦灵敏度分析来衡量每一层的敏感度，最后通过基于整数规划的方法进行位宽分配，与现有的量化方法相比，MixDQ 在保持 W8A8 品质的同时，实现了模型大小和内存成本的 3-4 倍减少，和 1.45 倍的延迟加速。

MixDQ: 高效节省内存的几步文本到图像扩散模型与度量分离混合精度量化

MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with  Metric-Decoupled Mixed Precision Quantization

Segment Anything Model (SAM) has shown impressive zero-shot transfer
performance for various computer vision tasks recently. However, its heavy
computation costs remain daunting for practical applications. MobileSAM
proposes to replace the heavyweight image encoder in SAM with TinyViT by
employing distillation, which results in a significant reduction in
computational requirements. However, its deployment on resource-constrained
mobile devices still encounters challenges due to the substantial memory and
computational overhead caused by self-attention mechanisms. Recently, RepViT
achieves the state-of-the-art performance and latency trade-off on mobile
devices by incorporating efficient architectural designs of ViTs into CNNs.
Here, to achieve real-time segmenting anything on mobile devices, following
MobileSAM, we replace the heavyweight image encoder in SAM with RepViT model,
ending up with the RepViT-SAM model. Extensive experiments show that RepViT-SAM
can enjoy significantly better zero-shot transfer capability than MobileSAM,
along with nearly $10\times$ faster inference speed. The code and models are
available at https://github.com/THU-MIG/RepViT.

为了在资源受限的移动设备上实现实时分割任何物体，本文使用了 RepViT-SAM 模型，代替了 SAM 中的重型图像编码器，得到了具有显著改进的零样本迁移能力和近 10 倍更快推理速度。