This paper presents a novel hybrid representation learning framework for
streaming data, where an image frame in a video is modeled by an ensemble of
two distinct deep neural networks; one is a low-bit quantized network and the
other is a lightweight full-precision network. The former learns coarse primary
information with low cost while the latter conveys residual information for
high fidelity to original representations. The proposed parallel architecture
is effective to maintain complementary information since fixed-point arithmetic
can be utilized in the quantized network and the lightweight model provides
precise representations given by a compact channel-pruned network. We
incorporate the hybrid representation technique into an online visual tracking
task, where deep neural networks need to handle temporal variations of target
appearances in real-time. Compared to the state-of-the-art real-time trackers
based on conventional deep neural networks, our tracking algorithm demonstrates
competitive accuracy on the standard benchmarks with a small fraction of
computational cost and memory footprint.

本文提出了一种新的混合表示学习框架，该框架可以针对流数据进行处理，其中视频中的图像帧由两个不同的深度神经网络集成模拟；一个是低位量化网络，另一个是轻量级全精度网络，前者以较低的成本学习粗略的主要信息，而后者则提供了能够还原表示的高保真度的剩余信息，所提出的并行结构是维护互补信息的有效方法，因为可以在量化网络中利用定点算术，并且轻量级模型提供了由紧凑通道修剪网络给出的精确表示，我们将混合表示技术纳入在线视觉跟踪任务，其中深度神经网络需要在实时处理目标外观的时间变化，与传统的深度神经网络实时跟踪器相比，在标准基准测试中，我们的跟踪算法表现出较高的准确性，只需很小的计算成本和内存占用。

在线轻量级混合表示学习：在视觉跟踪中的应用

Online Hybrid Lightweight Representations Learning: Its Application to  Visual Tracking

Deep neural networks, albeit their great success on feature learning in
various computer vision tasks, are usually considered as impractical for online
visual tracking because they require very long training time and a large number
of training samples. In this work, we present an efficient and very robust
tracking algorithm using a single Convolutional Neural Network (CNN) for
learning effective feature representations of the target object, in a purely
online manner. Our contributions are multifold: First, we introduce a novel
truncated structural loss function that maintains as many training samples as
possible and reduces the risk of tracking error accumulation. Second, we
enhance the ordinary Stochastic Gradient Descent approach in CNN training with
a robust sample selection mechanism. The sampling mechanism randomly generates
positive and negative samples from different temporal distributions, which are
generated by taking the temporal relations and label noise into account.
Finally, a lazy yet effective updating scheme is designed for CNN training.
Equipped with this novel updating algorithm, the CNN model is robust to some
long-existing difficulties in visual tracking such as occlusion or incorrect
detections, without loss of the effective adaption for significant appearance
changes. In the experiment, our CNN tracker outperforms all compared
state-of-the-art methods on two recently proposed benchmarks which in total
involve over 60 video sequences. The remarkable performance improvement over
the existing trackers illustrates the superiority of the feature
representations which are learned

本研究提出了一种利用单个卷积神经网络（CNN）实现在纯在线模式下学习目标对象有效特征表示的高效且非常强大的跟踪算法，其利用一种新颖的截断结构损失函数、稳健的样本选择机制和一种懒但有效的更新算法，使 CNN 在视觉跟踪中对遮挡或错误检测等难以处理的问题具有鲁棒性，在实验中，与现有跟踪器相比，该算法可以提高性能。