We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing
a 3D object from a few unposed images even with little visual overlap, while
simultaneously estimating the relative camera poses in ~1.3 seconds on a single
A100 GPU. PF-LRM is a highly scalable method utilizing the self-attention
blocks to exchange information between 3D object tokens and 2D image tokens; we
predict a coarse point cloud for each view, and then use a differentiable
Perspective-n-Point (PnP) solver to obtain camera poses. When trained on a huge
amount of multi-view posed data of ~1M objects, PF-LRM shows strong
cross-dataset generalization ability, and outperforms baseline methods by a
large margin in terms of pose prediction accuracy and 3D reconstruction quality
on various unseen evaluation datasets. We also demonstrate our model's
applicability in downstream text/image-to-3D task with fast feed-forward
inference. Our project website is at: this https URL .

我们提出了一种无姿势大重建模型（PF-LRM），可以从少数非姿势图像中重建三维物体，即使视觉重叠很少，同时在单个 A100 GPU 上估计相对相机姿势，仅需约 1.3 秒。PF-LRM 是一种高度可扩展的方法，利用自注意力块在三维物体标记和二维图像标记之间交换信息；我们为每个视图预测粗略的点云，然后使用可微的透视 - n - 点（PnP）求解器获得相机姿势。当在约 1M 个多视图姿势数据上进行训练时，PF-LRM 表现出强大的跨数据集泛化能力，并在各种未见评估数据集上以姿势预测准确性和三维重建质量大幅超越基线方法。我们还展示了模型在下游文本 / 图像到三维任务中的适用性，并具有快速的前馈推理。项目网站位于: this https URL。

PF-LRM：用于联合姿态和形状预测的无姿态大型重建模型

PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape  Prediction

The deployment of transformers for visual object tracking has shown
state-of-the-art results on several benchmarks. However, the transformer-based
models are under-utilized for Siamese lightweight tracking due to the
computational complexity of their attention blocks. This paper proposes an
efficient self and mixed attention transformer-based architecture for
lightweight tracking. The proposed backbone utilizes the separable mixed
attention transformers to fuse the template and search regions during feature
extraction to generate superior feature encoding. Our prediction head performs
global contextual modeling of the encoded features by leveraging efficient
self-attention blocks for robust target state estimation. With these
contributions, the proposed lightweight tracker deploys a transformer-based
backbone and head module concurrently for the first time. Our ablation study
testifies to the effectiveness of the proposed combination of backbone and head
modules. Simulations show that our Separable Self and Mixed Attention-based
Tracker, SMAT, surpasses the performance of related lightweight trackers on
GOT10k, TrackingNet, LaSOT, NfS30, UAV123, and AVisT datasets, while running at
37 fps on CPU, 158 fps on GPU, and having 3.8M parameters. For example, it
significantly surpasses the closely related trackers E.T.Track and
MixFormerV2-S on GOT10k-test by a margin of 7.9% and 5.8%, respectively, in the
AO metric. The tracker code and model is available at
this https URL

本文提出了一种用于轻量级追踪的高效自适应混合注意力变压器架构，实现了对相关轻量级追踪策略的显著改进，达到了先进的性能，在多个数据集上运行时帧率较高，参数量较小。