There is currently strong interest in improving visual object tracking by
augmenting the RGB modality with the output of a visual event camera that is
particularly informative about the scene motion. However, existing approaches
perform event feature extraction for RGB-E tracking using traditional
appearance models, which have been optimised for RGB only tracking, without
adapting it for the intrinsic characteristics of the event data. To address
this problem, we propose an Event backbone (Pooler), designed to obtain a
high-quality feature representation that is cognisant of the innate
characteristics of the event data, namely its sparsity. In particular,
Multi-Scale Pooling is introduced to capture all the motion feature trends
within event data through the utilisation of diverse pooling kernel sizes. The
association between the derived RGB and event representations is established by
an innovative module performing adaptive Mutually Guided Fusion (MGF).
Extensive experimental results show that our method significantly outperforms
state-of-the-art trackers on two widely used RGB-E tracking datasets, including
VisEvent and COESOT, where the precision and success rates on COESOT are
improved by 4.9% and 5.2%, respectively. Our code will be available at
this https URL

通过改进 RGB 模式的视觉对象跟踪，利用视觉事件相机的输出，特别是对场景运动特别有信息量的输出，本文通过引入一个适应事件数据固有特性的事件 backbone (Pooler)，提出了一种高质量的特征表示方法，该方法利用多尺度池化来捕捉事件数据中的所有运动特征趋势，并通过一个创新的模块进行自适应互相指导融合（MGF），在两个广泛使用的 RGB-E 跟踪数据集上，我们的方法在精确度和成功率上分别提高了 4.9% 和 5.2%。

TENet：融合多尺度池化和相互引导融合的 RGB-E 目标追踪

TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling  and Mutually-Guided Fusion for RGB-E Object Tracking

The primary bottleneck towards obtaining good recognition performance in IR
images is the lack of sufficient labeled training data, owing to the cost of
acquiring such data. Realizing that object detection methods for the RGB
modality are quite robust (at least for some commonplace classes, like person,
car, etc.), thanks to the giant training sets that exist, in this work we seek
to leverage cues from the RGB modality to scale object detectors to the IR
modality, while preserving model performance in the RGB modality. At the core
of our method, is a novel tensor decomposition method called TensorFact which
splits the convolution kernels of a layer of a Convolutional Neural Network
(CNN) into low-rank factor matrices, with fewer parameters than the original
CNN. We first pretrain these factor matrices on the RGB modality, for which
plenty of training data are assumed to exist and then augment only a few
trainable parameters for training on the IR modality to avoid over-fitting,
while encouraging them to capture complementary cues from those trained only on
the RGB modality. We validate our approach empirically by first assessing how
well our TensorFact decomposed network performs at the task of detecting
objects in RGB images vis-a-vis the original network and then look at how well
it adapts to IR images of the FLIR ADAS v1 dataset. For the latter, we train
models under scenarios that pose challenges stemming from data paucity. From
the experiments, we observe that: (i) TensorFact shows performance gains on RGB
images; (ii) further, this pre-trained model, when fine-tuned, outperforms a
standard state-of-the-art object detector on the FLIR ADAS v1 dataset by about
4% in terms of mAP 50 score.

通过从 RGB 模态中获取线索来扩展红外模态中的物体检测器，该方法利用了一种名为 TensorFact 的新型张量分解方法，通过预训练因子矩阵，进一步优化在 FLIR ADAS v1 数据集上的性能。

张量分解用于数据受限红外物体检测中的跨模态知识利用

Tensor Factorization for Leveraging Cross-Modal Knowledge in  Data-Constrained Infrared Object Detection

Cross-modality person re-identification is a challenging problem which
retrieves a given pedestrian image in RGB modality among all the gallery images
in infrared modality. The task can address the limitation of RGB-based person
Re-ID in dark environments. Existing researches mainly focus on enlarging
inter-class differences of feature to solve the problem. However, few studies
investigate improving intra-class cross-modality similarity, which is important
for this issue. In this paper, we propose a novel loss function, called
Hetero-Center loss (HC loss) to reduce the intra-class cross-modality
variations. Specifically, HC loss can supervise the network learning the
cross-modality invariant information by constraining the intra-class center
distance between two heterogenous modalities. With the joint supervision of
Cross-Entropy (CE) loss and HC loss, the network is trained to achieve two
vital objectives, inter-class discrepancy and intra-class cross-modality
similarity as much as possible. Besides, we propose a simple and
high-performance network architecture to learn local feature representations
for cross-modality person re-identification, which can be a baseline for future
research. Extensive experiments indicate the effectiveness of the proposed
methods, which outperform state-of-the-art methods by a wide margin.

本文提出了一种新的损失函数，叫做 Hetero-Center loss（HC loss），用于减少跨模态内类别的差异，并提出了一种简单高效的网络架构，用于学习跨模态人员重新识别的本地特征表示，得到了比现有方法更好的结果。