Diffusion Transformers (DiTs) are a powerful yet underexplored class of generative models compared to U-Net-based diffusion models. To bridge this gap, we introduce TIDE (Temporal-aware Sparse Autoencoders for Interpretable Diffusion transformErs), a novel framework that enhances temporal reconstruction within DiT activation layers across denoising steps. TIDE employs Sparse Autoencoders (SAEs) with a sparse bottleneck layer to extract interpretable and hierarchical features, revealing that diffusion models inherently learn hierarchical features at multiple levels (e.g., 3D, semantic, class) during generative pre-training. Our approach achieves state-of-the-art reconstruction performance, with a mean squared error (MSE) of 1e-3 and a cosine similarity of 0.97, demonstrating superior accuracy in capturing activation dynamics along the denoising trajectory. Beyond interpretability, we showcase TIDE's potential in downstream applications such as sparse activation-guided image editing and style transfer, enabling improved controllability for generative systems. By providing a comprehensive training and evaluation protocol tailored for DiTs, TIDE contributes to developing more interpretable, transparent, and trustworthy generative models.

该研究解决了扩散变换器（DiTs）在生成模型中的研究不足问题。提出的TIDE框架通过增强DiT激活层中的时序重建，利用稀疏自编码器提取可解释的层次特征，表明扩散模型在生成预训练中固有学习多层次特征。研究结果显示，该方法在重建性能上达到了最先进水平，并且在稀疏激活引导的图像编辑和风格转换中展现出良好的应用潜力。

时序感知稀疏自编码器用于可解释扩散变换器在图像生成中的应用