The process of generating data such as images is controlled by independent
and unknown factors of variation. The retrieval of these variables has been
studied extensively in the disentanglement, causal representation learning, and
independent component analysis fields. Recently, approaches merging these
domains together have shown great success. Instead of directly representing the
factors of variation, the problem of disentanglement can be seen as finding the
interventions on one image that yield a change to a single factor. Following
this assumption, we introduce a new method for disentanglement inspired by
causal dynamics that combines causality theory with vector-quantized
variational autoencoders. Our model considers the quantized vectors as causal
variables and links them in a causal graph. It performs causal interventions on
the graph and generates atomic transitions affecting a unique factor of
variation in the image. We also introduce a new task of action retrieval that
consists of finding the action responsible for the transition between two
images. We test our method on standard synthetic and real-world disentanglement
datasets. We show that it can effectively disentangle the factors of variation
and perform precise interventions on high-level semantic attributes of an image
without affecting its quality, even with imbalanced data distributions.

通过将因果关系理论与矢量量化变分自编码器相结合，我们提出了一种新的基于因果动态的去纠缠方法，将量化向量视为因果变量并将其链接在因果图中，通过对因果图进行因果干预，引发影响图像中唯一变化因素的原子转换。同时，我们通过引入一个新的行动检索任务来测试我们的模型。我们在标准的合成和真实世界的去纠缠数据集上进行了测试，结果表明它能够有效地解缠图像变化因素并对图像的高级语义属性进行精确干预，而不影响其质量，即使在不均衡的数据分布下也是如此。

通过稀疏因果干预分离潜在表示

Disentanglement of Latent Representations via Sparse Causal  Interventions

The human perception system is often assumed to recruit motor knowledge when
processing auditory speech inputs. Using articulatory modeling and deep
learning, this study examines how this articulatory information can be used for
discovering speech units in a self-supervised setting. We used vector-quantized
variational autoencoders (VQ-VAE) to learn discrete representations from
articulatory and acoustic speech data. In line with the zero-resource paradigm,
an ABX test was then used to investigate how the extracted representations
encode phonetically relevant properties. Experiments were conducted on three
different corpora in English and French. We found that articulatory information
rather organises the latent representations in terms of place of articulation
whereas the speech acoustics mainly structure the latent space in terms of
manner of articulation. We show that an optimal fusion of the two modalities
can lead to a joint representation of these phonetic dimensions more accurate
than each modality considered individually. Since articulatory information is
usually not available in a practical situation, we finally investigate the
benefit it provides when inferred from the speech acoustics in a
self-supervised manner.

使用关节建模和深度学习研究了如何在自我监督情况下使用关节信息发现语音单元，并在实验中发现联合使用关节和语音学模态可以更准确地表示这些语音学维度。