We propose 3Deformer, a general-purpose framework for interactive 3D shape
editing. Given a source 3D mesh with semantic materials, and a user-specified
semantic image, 3Deformer can accurately edit the source mesh following the
shape guidance of the semantic image, while preserving the source topology as
rigid as possible. Recent studies of 3D shape editing mostly focus on learning
neural networks to predict 3D shapes, which requires high-cost 3D training
datasets and is limited to handling objects involved in the datasets. Unlike
these studies, our 3Deformer is a non-training and common framework, which only
requires supervision of readily-available semantic images, and is compatible
with editing various objects unlimited by datasets. In 3Deformer, the source
mesh is deformed utilizing the differentiable renderer technique, according to
the correspondences between semantic images and mesh materials. However,
guiding complex 3D shapes with a simple 2D image incurs extra challenges, that
is, the deform accuracy, surface smoothness, geometric rigidity, and global
synchronization of the edited mesh should be guaranteed. To address these
challenges, we propose a hierarchical optimization architecture to balance the
global and local shape features, and propose further various strategies and
losses to improve properties of accuracy, smoothness, rigidity, and so on.
Extensive experiments show that our 3Deformer is able to produce impressive
results and reaches the state-of-the-art level.

我们提出了 3Deformer，这是一个用于交互式三维形状编辑的通用框架。通过利用语义图像对源网格进行形状引导，同时尽可能保持源拓扑结构的刚性，3Deformer 能够精确编辑源网格。我们的方法不需要高成本的三维训练数据集，只需利用易得的语义图像作为监督，并且可以在各种不受数据集限制的对象上进行编辑。通过引入分层优化架构和各种策略和损失函数，我们克服了使用简单的二维图像对复杂三维形状进行引导所带来的挑战，从而提高了准确性、平滑度和刚性。大量实验证明，我们的 3Deformer 能够产生令人印象深刻的结果，并达到了最先进水平。

3Deformer：一种基于图像引导的网格变形的通用框架

3Deformer: A Common Framework for Image-Guided Mesh Deformation

The Bilevel Optimization Problem is a hierarchical optimization problem with
two agents, a leader and a follower. The leader make their own decisions first,
and the followers make the best choices accordingly. The leader knows the
information of the followers, and the goal of the problem is to find the
optimal solution by considering the reactions of the followers from the
leader's point of view. For the Bilevel Optimization Problem, there are no
general and efficient algorithms or commercial solvers to get an optimal
solution, and it is very difficult to get a good solution even for a simple
problem. In this paper, we propose a deep learning approach using Graph Neural
Networks to solve the bilevel knapsack problem. We train the model to predict
the leader's solution and use it to transform the hierarchical optimization
problem into a single-level optimization problem to get the solution. Our model
found the feasible solution that was about 500 times faster than the exact
algorithm with $1.7\%$ optimal gap. Also, our model performed well on problems
of different size from the size it was trained on.

本文提出了使用图神经网络解决双层背包问题的深度学习方法，该方法通过训练模型来预测领导者的解决方案，从而将层次优化问题转化为单层优化问题，我们的模型找到的可行解比精确算法快了约 500 倍，并且在训练大小不同的问题上表现良好。