Visual object tracking, which is primarily based on visible light image
sequences, encounters numerous challenges in complicated scenarios, such as low
light conditions, high dynamic ranges, and background clutter. To address these
challenges, incorporating the advantages of multiple visual modalities is a
promising solution for achieving reliable object tracking. However, the
existing approaches usually integrate multimodal inputs through adaptive local
feature interactions, which cannot leverage the full potential of visual cues,
thus resulting in insufficient feature modeling. In this study, we propose a
novel multimodal hybrid tracker (MMHT) that utilizes frame-event-based data for
reliable single object tracking. The MMHT model employs a hybrid backbone
consisting of an artificial neural network (ANN) and a spiking neural network
(SNN) to extract dominant features from different visual modalities and then
uses a unified encoder to align the features across different domains.
Moreover, we propose an enhanced transformer-based module to fuse multimodal
features using attention mechanisms. With these methods, the MMHT model can
effectively construct a multiscale and multidimensional visual feature space
and achieve discriminative feature modeling. Extensive experiments demonstrate
that the MMHT model exhibits competitive performance in comparison with that of
other state-of-the-art methods. Overall, our results highlight the
effectiveness of the MMHT model in terms of addressing the challenges faced in
visual object tracking tasks.

本研究提出了一种新型的多模态混合跟踪器 (MMHT)，利用基于帧事件的数据进行可靠的单目标跟踪，通过使用人工神经网络（ANN）和脉冲神经网络（SNN）构建混合骨干，并使用增强的基于 Transformer 的模块通过注意机制融合多模态特征，构建多尺度和多维度的视觉特征空间，从而实现了有益的特征建模。实验结果表明，MMHT 模型在解决视觉目标跟踪任务中面临的挑战方面具有竞争力。

基于多模态混合特征提取和基于 Transformer 的融合的可靠物体跟踪

Reliable Object Tracking by Multimodal Hybrid Feature Extraction and  Transformer-Based Fusion

RGB-D salient object detection (SOD) is usually formulated as a problem of
classification or regression over two modalities, i.e., RGB and depth. Hence,
effective RGBD feature modeling and multi-modal feature fusion both play a
vital role in RGB-D SOD. In this paper, we propose a depth-sensitive RGB
feature modeling scheme using the depth-wise geometric prior of salient
objects. In principle, the feature modeling scheme is carried out in a
depth-sensitive attention module, which leads to the RGB feature enhancement as
well as the background distraction reduction by capturing the depth geometry
prior. Moreover, to perform effective multi-modal feature fusion, we further
present an automatic architecture search approach for RGB-D SOD, which does
well in finding out a feasible architecture from our specially designed
multi-modal multi-scale search space. Extensive experiments on seven standard
benchmarks demonstrate the effectiveness of the proposed approach against the
state-of-the-art.

本文提出了一种深度敏感的 RGB 特征建模方案，并使用深度几何先验来实现特征增强和背景分心减少。另外，我们还提出了一种自动架构搜索方法来进行 RGB-D 显着对象检测，取得了比现有技术更好的结果。

深度敏感关注和自动多模态融合的 RGB-D 显著性检测

Deep RGB-D Saliency Detection with Depth-Sensitive Attention and  Automatic Multi-Modal Fusion

Fully exploring correlation among points in point clouds is essential for
their feature modeling. This paper presents a novel end-to-end graph model,
named Point2Node, to represent a given point cloud. Point2Node can dynamically
explore correlation among all graph nodes from different levels, and adaptively
aggregate the learned features. Specifically, first, to fully explore the
spatial correlation among points for enhanced feature description, in a
high-dimensional node graph, we dynamically integrate the node's correlation
with self, local, and non-local nodes. Second, to more effectively integrate
learned features, we design a data-aware gate mechanism to self-adaptively
aggregate features at the channel level. Extensive experiments on various point
cloud benchmarks demonstrate that our method outperforms the state-of-the-art.

本研究提出了一个名为 Point2Node 的端对端图模型来表示给定的点云，Point2Node 可以动态地探索不同级别的所有图节点之间的关联，并自适应地聚合所学的特征。