Automated X-ray image segmentation would accelerate research and development
in diagnostic and interventional precision medicine. Prior efforts have
contributed task-specific models capable of solving specific image analysis
problems, but the utility of these models is restricted to their particular
task domain, and expanding to broader use requires additional data, labels, and
retraining efforts. Recently, foundation models (FMs) -- machine learning
models trained on large amounts of highly variable data thus enabling broad
applicability -- have emerged as promising tools for automated image analysis.
Existing FMs for medical image analysis focus on scenarios and modalities where
objects are clearly defined by visually apparent boundaries, such as surgical
tool segmentation in endoscopy. X-ray imaging, by contrast, does not generally
offer such clearly delineated boundaries or structure priors. During X-ray
image formation, complex 3D structures are projected in transmission onto the
imaging plane, resulting in overlapping features of varying opacity and shape.
To pave the way toward an FM for comprehensive and automated analysis of
arbitrary medical X-ray images, we develop FluoroSAM, a language-aligned
variant of the Segment-Anything Model, trained from scratch on 1.6M synthetic
X-ray images. FluoroSAM is trained on data including masks for 128 organ types
and 464 non-anatomical objects, such as tools and implants. In real X-ray
images of cadaveric specimens, FluoroSAM is able to segment bony anatomical
structures based on text-only prompting with 0.51 and 0.79 DICE with
point-based refinement, outperforming competing SAM variants for all
structures. FluoroSAM is also capable of zero-shot generalization to segmenting
classes beyond the training set thanks to its language alignment, which we
demonstrate for full lung segmentation on real chest X-rays.

通过 FluoroSAM 模型，该研究针对医学 X 光图像，基于文本提示，在解剖结构分割方面表现出色，并且具有零样本泛化能力，能够扩展到未经训练的图像类别，从而推进医学诊断和干预的自动化分析。

FluoroSAM：X 射线图像分割的语言对齐基础模型

FluoroSAM: A Language-aligned Foundation Model for X-ray Image  Segmentation

Low light enhancement has gained increasing importance with the rapid
development of visual creation and editing. However, most existing enhancement
algorithms are designed to homogeneously increase the brightness of images to a
pre-defined extent, limiting the user experience. To address this issue, we
propose Controllable Light Enhancement Diffusion Model, dubbed CLE Diffusion, a
novel diffusion framework to provide users with rich controllability. Built
with a conditional diffusion model, we introduce an illumination embedding to
let users control their desired brightness level. Additionally, we incorporate
the Segment-Anything Model (SAM) to enable user-friendly region
controllability, where users can click on objects to specify the regions they
wish to enhance. Extensive experiments demonstrate that CLE Diffusion achieves
competitive performance regarding quantitative metrics, qualitative results,
and versatile controllability. Project page:
https://yuyangyin.github.io/CLEDiffusion/

低光增强变得越来越重要；我们提出了一种能够提供丰富可控性的 Controllable Light Enhancement Diffusion Model（CLE Diffusion）扩散框架，利用条件扩散模型和照明嵌入来让用户控制所需的亮度水平，同时结合 Segment-Anything Model（SAM）进行用户友好的区域可控性，以实现出色的性能和多样化的可控性。

CLE 扩散：可控光增强扩散模型

CLE Diffusion: Controllable Light Enhancement Diffusion Model

In this work, we propose SAM3D, a novel framework that is able to predict
masks in 3D point clouds by leveraging the Segment-Anything Model (SAM) in RGB
images without further training or finetuning. For a point cloud of a 3D scene
with posed RGB images, we first predict segmentation masks of RGB images with
SAM, and then project the 2D masks into the 3D points. Later, we merge the 3D
masks iteratively with a bottom-up merging approach. At each step, we merge the
point cloud masks of two adjacent frames with the bidirectional merging
approach. In this way, the 3D masks predicted from different frames are
gradually merged into the 3D masks of the whole 3D scene. Finally, we can
optionally ensemble the result from our SAM3D with the over-segmentation
results based on the geometric information of the 3D scenes. Our approach is
experimented with ScanNet dataset and qualitative results demonstrate that our
SAM3D achieves reasonable and fine-grained 3D segmentation results without any
training or finetuning of SAM.

SAM3D 是一个创新的框架，通过利用 RGB 图像中的 Segment-Anything 模型而无需进一步训练或微调，能够在 3D 点云中预测掩模，首先预测具有 SAM 的 RGB 图像的分割掩模，然后将 2D 掩模投影到 3D 点中。最后，两个相邻帧的点云掩模以双向合并的方式进行合并，最终可以将不同帧预测出的 3D 掩模逐渐合并为整个 3D 场景的 3D 掩模，实验结果表明 SAM3D 能够在不需要对 SAM 进行训练或微调的情况下，实现合理和细粒度的 3D 分割结果。

SAM3D: 三维场景中物体分割

SAM3D: Segment Anything in 3D Scenes

Modern image inpainting systems, despite the significant progress, often
struggle with mask selection and holes filling. Based on Segment-Anything Model
(SAM), we make the first attempt to the mask-free image inpainting and propose
a new paradigm of ``clicking and filling'', which is named as Inpaint Anything
(IA). The core idea behind IA is to combine the strengths of different models
in order to build a very powerful and user-friendly pipeline for solving
inpainting-related problems. IA supports three main features: (i) Remove
Anything: users could click on an object and IA will remove it and smooth the
``hole'' with the context; (ii) Fill Anything: after certain objects removal,
users could provide text-based prompts to IA, and then it will fill the hole
with the corresponding generative content via driving AIGC models like Stable
Diffusion; (iii) Replace Anything: with IA, users have another option to retain
the click-selected object and replace the remaining background with the newly
generated scenes. We are also very willing to help everyone share and promote
new projects based on our Inpaint Anything (IA). Our codes are available at
this https URL

该论文提出了一种名为 Inpaint Anything 的新范例，支持去除、填充和替换图像内容，并基于 Segment-Anything Model 实现无需遮罩的图像修复。