The effective extraction of spatial-angular features plays a crucial role in
light field image super-resolution (LFSR) tasks, and the introduction of
convolution and Transformers leads to significant improvement in this area.
Nevertheless, due to the large 4D data volume of light field images, many
existing methods opted to decompose the data into a number of lower-dimensional
subspaces and perform Transformers in each sub-space individually. As a side
effect, these methods inadvertently restrict the self-attention mechanisms to a
One-to-One scheme accessing only a limited subset of LF data, explicitly
preventing comprehensive optimization on all spatial and angular cues. In this
paper, we identify this limitation as subspace isolation and introduce a novel
Many-to-Many Transformer (M2MT) to address it. M2MT aggregates angular
information in the spatial subspace before performing the self-attention
mechanism. It enables complete access to all information across all
sub-aperture images (SAIs) in a light field image. Consequently, M2MT is
enabled to comprehensively capture long-range correlation dependencies. With
M2MT as the pivotal component, we develop a simple yet effective M2MT network
for LFSR. Our experimental results demonstrate that M2MT achieves
state-of-the-art performance across various public datasets. We further conduct
in-depth analysis using local attribution maps (LAM) to obtain visual
interpretability, and the results validate that M2MT is empowered with a truly
non-local context in both spatial and angular subspaces to mitigate subspace
isolation and acquire effective spatial-angular representation.

通过引入卷积和 Transformer，本文提出了一种新颖的 Many-to-Many Transformer（M2MT）算法，以解决光场图像超分辨率问题中的低维空间隔离限制，实现了跨子孔径图像的全局优化，从而显著改善了性能。