Recent advancements in diffusion models have introduced fast sampling methods
that can effectively produce high-quality images in just one or a few denoising
steps. Interestingly, when these are distilled from existing diffusion models,
they often maintain alignment with the original model, retaining similar
outputs for similar prompts and seeds. These properties present opportunities
to leverage fast sampling methods as a shortcut-mechanism, using them to create
a preview of denoised outputs through which we can backpropagate image-space
losses. In this work, we explore the potential of using such
shortcut-mechanisms to guide the personalization of text-to-image models to
specific facial identities. We focus on encoder-based personalization
approaches, and demonstrate that by tuning them with a lookahead identity loss,
we can achieve higher identity fidelity, without sacrificing layout diversity
or prompt alignment. We further explore the use of attention sharing mechanisms
and consistent data generation for the task of personalization, and find that
encoder training can benefit from both.

通过使用快速采样方法以及对编码器进行个性化训练，本研究探索了将快速采样方法用于文本到图像模型个性化，并通过预测身份目标让模型在保持多样性和对齐性的同时提高身份准确性的潜力。同时，研究还发现注意力共享机制和一致的数据生成对于编码器训练都有益处。

基于编码器的文本到图像个性化的 LCM-Lookahead

LCM-Lookahead for Encoder-based Text-to-Image Personalization

Graph contrastive learning (GCL) has become a powerful tool for learning
graph data, but its scalability remains a significant challenge. In this work,
we propose a simple yet effective training framework called Structural
Compression (StructComp) to address this issue. Inspired by a sparse low-rank
approximation on the diffusion matrix, StructComp trains the encoder with the
compressed nodes. This allows the encoder not to perform any message passing
during the training stage, and significantly reduces the number of sample pairs
in the contrastive loss. We theoretically prove that the original GCL loss can
be approximated with the contrastive loss computed by StructComp. Moreover,
StructComp can be regarded as an additional regularization term for GCL models,
resulting in a more robust encoder. Empirical studies on seven benchmark
datasets show that StructComp greatly reduces the time and memory consumption
while improving model performance compared to the vanilla GCL models and
scalable training methods.

通过稀疏低秩近似方法，提出了一种名为 StructComp 的简单而有效的训练框架，用于解决图形对比学习的可扩展性问题，并在理论上证明了 StructComp 可以近似原始的对比损失，并改进了模型性能。