Non-autoregressive GAN-based neural vocoders are widely used due to their
fast inference speed and high perceptual quality. However, they often suffer
from audible artifacts such as tonal artifacts in their generated results.
Therefore, we propose JenGAN, a new training strategy that involves stacking
shifted low-pass filters to ensure the shift-equivariant property. This method
helps prevent aliasing and reduce artifacts while preserving the model
structure used during inference. In our experimental evaluation, JenGAN
consistently enhances the performance of vocoder models, yielding significantly
superior scores across the majority of evaluation metrics.

JenGAN 是一种新的训练策略，通过堆叠位移的低通滤波器以确保位移等变性，从而提高非自回归基于 GAN 的神经声码器模型的性能并减少音调等人工痕迹。

JenGAN：基于 GAN 的语音合成中的堆叠偏移滤波器

JenGAN: Stacked Shifted Filters in GAN-Based Speech Synthesis

Transformer-based models have recently become wildly successful across a
diverse set of domains. At the same time, recent work has shown that
Transformers are inherently low-pass filters that gradually oversmooth the
inputs, reducing the expressivity of their representations. A natural question
is: How can Transformers achieve these successes given this shortcoming? In
this work we show that in fact Transformers are not inherently low-pass
filters. Instead, whether Transformers oversmooth or not depends on the
eigenspectrum of their update equations. Our analysis extends prior work in
oversmoothing and in the closely-related phenomenon of rank collapse. We show
that many successful Transformer models have attention and weights which
satisfy conditions that avoid oversmoothing. Based on this analysis, we derive
a simple way to parameterize the weights of the Transformer update equations
that allows for control over its spectrum, ensuring that oversmoothing does not
occur. Compared to a recent solution for oversmoothing, our approach improves
generalization, even when training with more layers, fewer datapoints, and data
that is corrupted.

Transformer 模型在不同领域取得了巨大的成功，但近期的研究发现，Transformer 本质上是低通滤波器，会逐渐过度平滑输入，从而降低其表达能力。然而，根据本研究，Transformer 是否过度平滑取决于其更新方程的特征频谱，因此提出了一种简洁的参数化方法，能够控制 Transformer 的频谱，避免过度平滑。相比于近期的解决方法，该方法提高了泛化性能，即使在更多层次、更少数据点和受损数据的训练情况下仍然有效。

Transformer 过度平滑的真相

Setting the Record Straight on Transformer Oversmoothing

Recent studies show that self-attentions behave like low-pass filters (as
opposed to convolutions) and enhancing their high-pass filtering capability
improves model performance. Contrary to this idea, we investigate existing
convolution-based models with spectral analysis and observe that improving the
low-pass filtering in convolution operations also leads to performance
improvement. To account for this observation, we hypothesize that utilizing
optimal token mixers that capture balanced representations of both high- and
low-frequency components can enhance the performance of models. We verify this
by decomposing visual features into the frequency domain and combining them in
a balanced manner. To handle this, we replace the balancing problem with a mask
filtering problem in the frequency domain. Then, we introduce a novel
token-mixer named SPAM and leverage it to derive a MetaFormer model termed as
SPANet. Experimental results show that the proposed method provides a way to
achieve this balance, and the balanced representations of both high- and
low-frequency components can improve the performance of models on multiple
computer vision tasks. Our code is available at
$\href{https://doranlyong.github.io/projects/spanet/}{\text{this https URL}}$.

使用自我关注机制、低通滤波器、基于卷积的模型、频谱分析和令牌混合技术来提高计算机视觉任务模型性能。

SPANet：频率平衡的标记混合器，使用谱池聚合调制

SPANet: Frequency-balancing Token Mixer using Spectral Pooling  Aggregation Modulation

Graph convolutional networks (GCNs) have recently enabled a popular class of
algorithms for collaborative filtering (CF). Nevertheless, the theoretical
underpinnings of their empirical successes remain elusive. In this paper, we
endeavor to obtain a better understanding of GCN-based CF methods via the lens
of graph signal processing. By identifying the critical role of smoothness, a
key concept in graph signal processing, we develop a unified graph
convolution-based framework for CF. We prove that many existing CF methods are
special cases of this framework, including the neighborhood-based methods,
low-rank matrix factorization, linear auto-encoders, and LightGCN,
corresponding to different low-pass filters. Based on our framework, we then
present a simple and computationally efficient CF baseline, which we shall
refer to as Graph Filter based Collaborative Filtering (GF-CF). Given an
implicit feedback matrix, GF-CF can be obtained in a closed form instead of
expensive training with back-propagation. Experiments will show that GF-CF
achieves competitive or better performance against deep learning-based methods
on three well-known datasets, notably with a $70\%$ performance gain over
LightGCN on the Amazon-book dataset.

本文通过图信号处理的角度，研究了基于图卷积网络 (GCNs) 的协作过滤 (CF) 方法的理论，提出了基于图卷积的统一框架，证明了现有的许多 CF 方法都是这个框架的特例，包括邻域方法、低秩矩阵分解、线性自编码器和 LightGCN 等，提出了基于图滤波的协作过滤 (GF-CF) 作为基准模型，在三个知名数据集上实验表明，GF-CF 与基于深度学习的方法相比性能更强，在 Amazon-book 数据集上比 LightGCN 的性能提升了 70%。