Large foundation models have recently emerged as a prominent focus of
interest, attaining superior performance in widespread scenarios. Due to the
scarcity of 3D data, many efforts have been made to adapt pre-trained
transformers from vision to 3D domains. However, such 2D-to-3D approaches are
still limited, due to the potential loss of spatial geometries and high
computation cost. More importantly, their frameworks are mainly designed for 2D
models, lacking a general any-to-3D paradigm. In this paper, we introduce
Any2Point, a parameter-efficient method to empower any-modality large models
(vision, language, audio) for 3D understanding. Given a frozen transformer from
any source modality, we propose a 3D-to-any (1D or 2D) virtual projection
strategy that correlates the input 3D points to the original 1D or 2D positions
within the source modality. This mechanism enables us to assign each 3D token
with a positional encoding paired with the pre-trained model, which avoids 3D
geometry loss caused by the true projection and better motivates the
transformer for 3D learning with 1D/2D positional priors. Then, within each
transformer block, we insert an any-to-3D guided adapter module for
parameter-efficient fine-tuning. The adapter incorporates prior spatial
knowledge from the source modality to guide the local feature aggregation of 3D
tokens, compelling the semantic adaption of any-modality transformers. We
conduct extensive experiments to showcase the effectiveness and efficiency of
our method. Code and models are released at
this https URL

Any2Point 是一种参数高效的方法，通过引入虚拟投影策略和引导适配器模块，使得基于任何模态（视觉、语言、音频）的大型模型能够进行 3D 理解，并且在实验中展示了方法的有效性和高效性。

Any2Point：强化各模态大型模型以实现高效的三维理解

Any2Point: Empowering Any-modality Large Models for Efficient 3D  Understanding

Procedural Content Generation via Reinforcement Learning (PCGRL) foregoes the
need for large human-authored data-sets and allows agents to train explicitly
on functional constraints, using computable, user-defined measures of quality
instead of target output. We explore the application of PCGRL to 3D domains, in
which content-generation tasks naturally have greater complexity and potential
pertinence to real-world applications. Here, we introduce several PCGRL tasks
for the 3D domain, Minecraft (Mojang Studios, 2009). These tasks will challenge
RL-based generators using affordances often found in 3D environments, such as
jumping, multiple dimensional movement, and gravity. We train an agent to
optimize each of these tasks to explore the capabilities of previous research
in PCGRL. This agent is able to generate relatively complex and diverse levels,
and generalize to random initial states and control targets. Controllability
tests in the presented tasks demonstrate their utility to analyze success and
failure for 3D generators.

使用强化学习实现程序化内容生成（PCGRL），通过可计算、用户定义的质量度量训练智能体优化 3D 环境下针对 Minecraft 游戏的任务，生成多样化的游戏关卡，展示了 PCGRL 在 3D 环境下的应用和潜力。