Noisy multi-label learning has garnered increasing attention due to the
challenges posed by collecting large-scale accurate labels, making noisy labels
a more practical alternative. Motivated by noisy multi-class learning, the
introduction of transition matrices can help model multi-label noise and enable
the development of statistically consistent algorithms for noisy multi-label
learning. However, estimating multi-label noise transition matrices remains a
challenging task, as most existing estimators in noisy multi-class learning
rely on anchor points and accurate fitting of noisy class posteriors, which is
hard to satisfy in noisy multi-label learning. In this paper, we address this
problem by first investigating the identifiability of class-dependent
transition matrices in noisy multi-label learning. Building upon the
identifiability results, we propose a novel estimator that leverages label
correlations without the need for anchor points or precise fitting of noisy
class posteriors. Specifically, we first estimate the occurrence probability of
two noisy labels to capture noisy label correlations. Subsequently, we employ
sample selection techniques to extract information implying clean label
correlations, which are then used to estimate the occurrence probability of one
noisy label when a certain clean label appears. By exploiting the mismatches in
label correlations implied by these occurrence probabilities, we demonstrate
that the transition matrix becomes identifiable and can be acquired by solving
a bilinear decomposition problem. Theoretically, we establish an estimation
error bound for our multi-label transition matrix estimator and derive a
generalization error bound for our statistically consistent algorithm.
Empirically, we validate the effectiveness of our estimator in estimating
multi-label noise transition matrices, leading to excellent classification
performance.

本文研究了噪声多标签学习中类相关的转换矩阵的可辨识性，并提出了一种新的估计器，该估计器利用标签相关性而无需锚点或精确拟合噪声类后验概率。通过信息提取纯净标签相关性，并利用这些事件概率暗示的标签相关性的差异，证明了转换矩阵的可辨识性，并通过解决双线性分解问题获得转换矩阵。理论上，我们建立了多标签转换矩阵估计器的估计误差界限，并推导出我们的统计一致算法的泛化误差界限。实验上，我们验证了我们的估计器在估计多标签噪声转换矩阵方面的有效性，从而导致了出色的分类性能。

多标签噪声转移矩阵估计与标签相关性：理论与算法

Multi-Label Noise Transition Matrix Estimation with Label Correlations:  Theory and Algorithm

Denoising diffusion probabilistic models (DDPMs) (Ho et al. 2020) have shown
impressive results on image and waveform generation in continuous state spaces.
Here, we introduce Discrete Denoising Diffusion Probabilistic Models (D3PMs),
diffusion-like generative models for discrete data that generalize the
multinomial diffusion model of Hoogeboom et al. 2021, by going beyond
corruption processes with uniform transition probabilities. This includes
corruption with transition matrices that mimic Gaussian kernels in continuous
space, matrices based on nearest neighbors in embedding space, and matrices
that introduce absorbing states. The third allows us to draw a connection
between diffusion models and autoregressive and mask-based generative models.
We show that the choice of transition matrix is an important design decision
that leads to improved results in image and text domains. We also introduce a
new loss function that combines the variational lower bound with an auxiliary
cross entropy loss. For text, this model class achieves strong results on
character-level text generation while scaling to large vocabularies on LM1B. On
the image dataset CIFAR-10, our models approach the sample quality and exceed
the log-likelihood of the continuous-space DDPM model.

本文提出了离散去噪扩散概率模型（D3PM），用于离散数据的扩散式生成模型，包括了仿真高斯核、嵌入空间中基于最邻近、引入吸收状态等过渡矩阵。研究表明过渡矩阵的选择对图像和文本领域下的生成模型结果至关重要，且提出的新损失函数在字符级别文本生成上取得了很好的效果。

离散状态空间中的结构去噪扩散模型

Structured Denoising Diffusion Models in Discrete State-Spaces

We study the problem of learning overcomplete HMMs---those that have many
hidden states but a small output alphabet. Despite having significant practical
importance, such HMMs are poorly understood with no known positive or negative
results for efficient learning. In this paper, we present several new
results---both positive and negative---which help define the boundaries between
the tractable and intractable settings. Specifically, we show positive results
for a large subclass of HMMs whose transition matrices are sparse,
well-conditioned, and have small probability mass on short cycles. On the other
hand, we show that learning is impossible given only a polynomial number of
samples for HMMs with a small output alphabet and whose transition matrices are
random regular graphs with large degree. We also discuss these results in the
context of learning HMMs which can capture long-term dependencies.

本文研究了过完备的 HMMs 学习问题，提出了正负两方面的新结果，界定了可处理和不可处理问题之间的边界，并讨论了如何学习能够捕捉长期依赖关系的过完备 HMMs。