Volumetric optical microscopy using non-diffracting beams enables rapid
imaging of 3D volumes by projecting them axially to 2D images but lacks crucial
depth information. Addressing this, we introduce MicroDiffusion, a pioneering
tool facilitating high-quality, depth-resolved 3D volume reconstruction from
limited 2D projections. While existing Implicit Neural Representation (INR)
models often yield incomplete outputs and Denoising Diffusion Probabilistic
Models (DDPM) excel at capturing details, our method integrates INR's
structural coherence with DDPM's fine-detail enhancement capabilities. We
pretrain an INR model to transform 2D axially-projected images into a
preliminary 3D volume. This pretrained INR acts as a global prior guiding
DDPM's generative process through a linear interpolation between INR outputs
and noise inputs. This strategy enriches the diffusion process with structured
3D information, enhancing detail and reducing noise in localized 2D images. By
conditioning the diffusion model on the closest 2D projection, MicroDiffusion
substantially enhances fidelity in resulting 3D reconstructions, surpassing INR
and standard DDPM outputs with unparalleled image quality and structural
fidelity. Our code and dataset are available at
this https URL

通过使用非衍射光束进行体积光学显微镜法，我们提出了 MicroDiffusion 工具，用于从有限的 2D 投影中进行高质量的、深度解析的 3D 体积重建。该方法融合了 INR 模型的结构连贯性和 DDPM 模型的细节增强能力，通过将 INR 和噪声输入之间的线性插值来指导 DDPM 的生成过程，从而提高了扩散过程的结构化 3D 信息，增强了本地化 2D 图像的细节，并在生成的 3D 重建中显著提高了保真度。

微扩散：由有限 2D 显微投影重建的隐式表示引导的 3D 重建

MicroDiffusion: Implicit Representation-Guided Diffusion for 3D  Reconstruction from Limited 2D Microscopy Projections

Electron microscopy (EM) images exhibit anisotropic axial resolution due to
the characteristics inherent to the imaging modality, presenting challenges in
analysis and downstream tasks.In this paper, we propose a diffusion-model-based
framework that overcomes the limitations of requiring reference data or prior
knowledge about the degradation process. Our approach utilizes 2D diffusion
models to consistently reconstruct 3D volumes and is well-suited for highly
downsampled data. Extensive experiments conducted on two public datasets
demonstrate the robustness and superiority of leveraging the generative prior
compared to supervised learning methods. Additionally, we demonstrate our
method's feasibility for self-supervised reconstruction, which can restore a
single anisotropic volume without any training data.

基于扩散模型的框架实现了逆转因成像模态的特性而导致各向异性轴向分辨率的挑战，且无需参考数据或先验知识，通过利用 2D 扩散模型持续重建 3D 体积，在高度下采样数据中表现出优越性，并通过在两个公共数据集上的实验，证明了与监督学习方法相比利用生成先验的鲁棒性和优势。此外，还证实了自监督重建方法可以在无训练数据的情况下恢复单个各向异性体积。

无参考各向同性三维电子显微重建

Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models

As many different 3D volumes could produce the same 2D x-ray image, inverting
this process is challenging. We show that recent deep learning-based
convolutional neural networks can solve this task. As the main challenge in
learning is the sheer amount of data created when extending the 2D image into a
3D volume, we suggest firstly to learn a coarse, fixed-resolution volume which
is then fused in a second step with the input x-ray into a high-resolution
volume. To train and validate our approach we introduce a new dataset that
comprises of close to half a million computer-simulated 2D x-ray images of 3D
volumes scanned from 175 mammalian species. Applications of our approach
include stereoscopic rendering of legacy x-ray images, re-rendering of x-rays
including changes of illumination, view pose or geometry. Our evaluation
includes comparison to previous tomography work, previous learning methods
using our data, a user study and application to a set of real x-rays.

本研究展示了基于深度学习的卷积神经网络可以重构二维 X 光成为三维体积，通过将粗略的，固定分辨率的体积与输入的 X 光融合成高分辨率的体积，以实现更加精细化的成像。同时，研究团队还创建了一个新的数据集，其中包括了将近 50 万个 175 个哺乳动物物种的计算机模拟二维 X 光成像的数据，为以前基于层析成像和数据驱动学习的方法提供了比较和验证，并应用于实际 X 光成像数据的重构和渲染。