3D generation has raised great attention in recent years. With the success of
text-to-image diffusion models, the 2D-lifting technique becomes a promising
route to controllable 3D generation. However, these methods tend to present
inconsistent geometry, which is also known as the Janus problem. We observe
that the problem is caused mainly by two aspects, i.e., viewpoint bias in 2D
diffusion models and overfitting of the optimization objective. To address it,
we propose a two-stage 2D-lifting framework, namely DreamControl, which
optimizes coarse NeRF scenes as 3D self-prior and then generates fine-grained
objects with control-based score distillation. Specifically, adaptive viewpoint
sampling and boundary integrity metric are proposed to ensure the consistency
of generated priors. The priors are then regarded as input conditions to
maintain reasonable geometries, in which conditional LoRA and weighted score
are further proposed to optimize detailed textures. DreamControl can generate
high-quality 3D content in terms of both geometry consistency and texture
fidelity. Moreover, our control-based optimization guidance is applicable to
more downstream tasks, including user-guided generation and 3D animation. The
project page is available at this https URL

通过 DreamControl 的两阶段 2D-lifting 框架，实现了在 3D 生成中解决几何不一致性的 Janus 问题，并生成出几何一致性和纹理保真度都很高的高质量 3D 内容。

DreamControl: 基于控制的文本到 3D 生成与 3D 自先

DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior

Feature matching is a crucial technique in computer vision. Essentially, it
can be considered as a searching problem to establish correspondences between
images. The key challenge in this task lies in the lack of a well-defined
search space, leading to inaccurate point matching of current methods. In
pursuit of a reasonable matching search space, this paper introduces a
hierarchical feature matching framework: Area to Point Matching (A2PM), to
first find semantic area matches between images, and then perform point
matching on area matches, thus setting the search space as the area matches
with salient features to achieve high matching precision. This proper search
space of A2PM framework also alleviates the accuracy limitation in
state-of-the-art Transformer-based matching methods. To realize this framework,
we further propose Semantic and Geometry Area Matching (SGAM) method, which
utilizes semantic prior and geometry consistency to establish accurate area
matches between images. By integrating SGAM with off-the-shelf
Transformer-based matchers, our feature matching methods, adopting the A2PM
framework, achieve encouraging precision improvements in massive point matching
and pose estimation experiments for present arts.

本文介绍了一种基于面积匹配的层次型特征匹配框架：A2PM，通过先找到图像间语义区域匹配，再在区域匹配上执行点匹配，将搜索空间设置为具有显著特征的区域匹配，从而实现高匹配精度，并使用语义和几何区域匹配方法（SGAM）来建立精确无误的区域匹配。通过将 SGAM 与现有的 Transformer-based 匹配器结合使用，我们的特征匹配方法在大规模点匹配和姿态估计实验中取得了令人鼓舞的精确度改进。

从区域到点的搜索：面向语义几何特征匹配的层次框架

Searching from Area to Point: A Hierarchical Framework for  Semantic-Geometric Combined Feature Matching

Unified visual grounding pursues a simple and generic technical route to
leverage multi-task data with less task-specific design. The most advanced
methods typically present boxes and masks as vertex sequences to model
referring detection and segmentation as an autoregressive sequential vertex
generation paradigm. However, generating high-dimensional vertex sequences
sequentially is error-prone because the upstream of the sequence remains static
and cannot be refined based on downstream vertex information, even if there is
a significant location gap. Besides, with limited vertexes, the inferior
fitting of objects with complex contours restricts the performance upper bound.
To deal with this dilemma, we propose a parallel vertex generation paradigm for
superior high-dimension scalability with a diffusion model by simply modifying
the noise dimension. An intuitive materialization of our paradigm is Parallel
Vertex Diffusion (PVD) to directly set vertex coordinates as the generation
target and use a diffusion model to train and infer. We claim that it has two
flaws: (1) unnormalized coordinate caused a high variance of loss value; (2)
the original training objective of PVD only considers point consistency but
ignores geometry consistency. To solve the first flaw, Center Anchor Mechanism
(CAM) is designed to convert coordinates as normalized offset values to
stabilize the training loss value. For the second flaw, Angle summation loss
(ASL) is designed to constrain the geometry difference of prediction and ground
truth vertexes for geometry-level consistency. Empirical results show that our
PVD achieves state-of-the-art in both referring detection and segmentation, and
our paradigm is more scalable and efficient than sequential vertex generation
with high-dimension data.

本文提出了一种并行顶点生成模型，采用扩散模型训练，并引入归一化坐标值和角度损失以解决训练的两个缺陷，实验结果显示该模型在物体检测和分割方面取得了最先进的性能。