We present iSeg, a new interactive technique for segmenting 3D shapes.
Previous works have focused mainly on leveraging pre-trained 2D foundation
models for 3D segmentation based on text. However, text may be insufficient for
accurately describing fine-grained spatial segmentations. Moreover, achieving a
consistent 3D segmentation using a 2D model is challenging since occluded areas
of the same semantic region may not be visible together from any 2D view. Thus,
we design a segmentation method conditioned on fine user clicks, which operates
entirely in 3D. Our system accepts user clicks directly on the shape's surface,
indicating the inclusion or exclusion of regions from the desired shape
partition. To accommodate various click settings, we propose a novel
interactive attention module capable of processing different numbers and types
of clicks, enabling the training of a single unified interactive segmentation
model. We apply iSeg to a myriad of shapes from different domains,
demonstrating its versatility and faithfulness to the user's specifications.
Our project page is at this https URL

我们提出了一种名为 iSeg 的新型交互式技术，用于对 3D 形状进行分割。

iSeg: 基于交互式注意力的交互式 3D 分割

iSeg: Interactive 3D Segmentation via Interactive Attention

A large body of recent work targets semantically conditioned image
generation. Most such methods focus on the narrower task of pose transfer and
ignore the more challenging task of subject transfer that consists in not only
transferring the pose but also the appearance and background. In this work, we
introduce SCAM (Semantic Cross Attention Modulation), a system that encodes
rich and diverse information in each semantic region of the image (including
foreground and background), thus achieving precise generation with emphasis on
fine details. This is enabled by the Semantic Attention Transformer Encoder
that extracts multiple latent vectors for each semantic region, and the
corresponding generator that exploits these multiple latents by using semantic
cross attention modulation. It is trained only using a reconstruction setup,
while subject transfer is performed at test time. Our analysis shows that our
proposed architecture is successful at encoding the diversity of appearance in
each semantic region. Extensive experiments on the iDesigner and CelebAMask-HD
datasets show that SCAM outperforms SEAN and SPADE; moreover, it sets the new
state of the art on subject transfer.

本文提出了一个名为 SCAM (Semantic Cross Attention Modulation) 的系统，通过语义交叉注意力调节实现了精准图像生成，该系统可用于主体迁移的更具挑战性的任务，并在 iDesigner 和 CelebAMask-HD 数据集上超越了现有的 SEAN 和 SPADE 方法。

SCAM！基于语义交叉注意力调节的图像中人物迁移

SCAM! Transferring humans between images with Semantic Cross Attention Modulation

Most existing style transfer methods follow the assumption that styles can be
represented with global statistics (e.g., Gram matrices or covariance
matrices), and thus address the problem by forcing the output and style images
to have similar global statistics. An alternative is the assumption of local
style patterns, where algorithms are designed to swap similar local features of
content and style images. However, the limitation of these existing methods is
that they neglect the semantic structure of the content image which may lead to
corrupted content structure in the output. In this paper, we make a new
assumption that image features from the same semantic region form a manifold
and an image with multiple semantic regions follows a multi-manifold
distribution. Based on this assumption, the style transfer problem is
formulated as aligning two multi-manifold distributions and a Manifold
Alignment based Style Transfer (MAST) framework is proposed. The proposed
framework allows semantically similar regions between the output and the style
image share similar style patterns. Moreover, the proposed manifold alignment
method is flexible to allow user editing or using semantic segmentation maps as
guidance for style transfer. To allow the method to be applicable to
photorealistic style transfer, we propose a new adaptive weight skip connection
network structure to preserve the content details. Extensive experiments verify
the effectiveness of the proposed framework for both artistic and
photorealistic style transfer. Code is available at
this https URL

本文提出了基于流形对齐的样式转换方法（MAST），通过假设图像的特征分布符合多重流形分布的假设，可以学习到图像的语义区域与样式模式的匹配关系，通过自适应权重跳跃连接网络结构来保留内容的细节，从而实现艺术风格和真实感风格的转换。