Token filtering to reduce irrelevant tokens prior to self-attention is a
straightforward way to enable efficient vision Transformer. This is the first
work to view token filtering from a feature selection perspective, where we
weigh the importance of a token according to how much it can change the loss
once masked. If the loss changes greatly after masking a token of interest, it
means that such a token has a significant impact on the final decision and is
thus relevant. Otherwise, the token is less important for the final decision,
so it can be filtered out. After applying the token filtering module
generalized from the whole training data, the token number fed to the
self-attention module can be obviously reduced in the inference phase, leading
to much fewer computations in all the subsequent self-attention layers. The
token filter can be realized using a very simple network, where we utilize
multi-layer perceptron. Except for the uniqueness of performing token filtering
only once from the very beginning prior to self-attention, the other core
feature making our method different from the other token filters lies in the
predictability of token impact from a feature selection point of view. The
experiments show that the proposed method provides an efficient way to approach
a light weighted model after optimized with a backbone by means of fine tune,
which is easy to be deployed in comparison with the existing methods based on
training from scratch.

本文是一项利用特征选择视角看待标记过滤技术，以达到降低自注意力计算量优化模型的研究；研究表明，该方法结合主干网络的优化微调可实现轻量级模型的可以快速部署。

预测 Token 对高效视觉 Transformer 的影响

Predicting Token Impact Towards Efficient Vision Transformer

RGB-D salient object detection (SOD) recently has attracted increasing
research interest by benefiting conventional RGB SOD with extra depth
information. However, existing RGB-D SOD models often fail to perform well in
terms of both efficiency and accuracy, which hinders their potential
applications on mobile devices and real-world problems. An underlying challenge
is that the model accuracy usually degrades when the model is simplified to
have few parameters. To tackle this dilemma and also inspired by the fact that
depth quality is a key factor influencing the accuracy, we propose a novel
depth quality-inspired feature manipulation (DQFM) process, which is efficient
itself and can serve as a gating mechanism for filtering depth features to
greatly boost the accuracy. DQFM resorts to the alignment of low-level RGB and
depth features, as well as holistic attention of the depth stream to explicitly
control and enhance cross-modal fusion. We embed DQFM to obtain an efficient
light-weight model called DFM-Net, where we also design a tailored depth
backbone and a two-stage decoder for further efficiency consideration.
Extensive experimental results demonstrate that our DFM-Net achieves
state-of-the-art accuracy when comparing to existing non-efficient models, and
meanwhile runs at 140ms on CPU (2.2$\times$ faster than the prior fastest
efficient model) with only $\sim$8.5Mb model size (14.9% of the prior
lightest). Our code will be available at this https URL

本文提出了一种基于深度图像质量的特征处理方法（DQFM）和一种高效的轻量级模型（DFM-Net），用于 RGB-D 显着目标检测，通过跨模态融合来增强深度质量，并在非常高效的情况下实现了最先进的精度和性能。

基于深度品质启发的特征操作，用于高效的 RGB-D 显著性对象检测

Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient  Object Detection

Click-through rate (CTR) prediction plays a critical role in recommender
systems and online advertising. The data used in these applications are
multi-field categorical data, where each feature belongs to one field. Field
information is proved to be important and there are several works considering
fields in their models. In this paper, we proposed a novel approach to model
the field information effectively and efficiently. The proposed approach is a
direct improvement of FwFM, and is named as Field-matrixed Factorization
Machines (FmFM, or $FM^2$). We also proposed a new explanation of FM and FwFM
within the FmFM framework, and compared it with the FFM. Besides pruning the
cross terms, our model supports field-specific variable dimensions of embedding
vectors, which acts as soft pruning. We also proposed an efficient way to
minimize the dimension while keeping the model performance. The FmFM model can
also be optimized further by caching the intermediate vectors, and it only
takes thousands of floating-point operations (FLOPs) to make a prediction. Our
experiment results show that it can out-perform the FFM, which is more complex.
The FmFM model's performance is also comparable to DNN models which require
much more FLOPs in runtime.

该研究提出了一种新的方法，名为 Field-matrixed Factorization Machines（FmFM，或 FM^2），用于有效和高效地建模数据中的字段信息，在交叉项剪枝的基础上支持字段特定变量维度的嵌入向量，并通过缓存中间向量进一步优化模型，实验结果显示该方法可以优于 FFM 和 DNN 模型。

$FM^2$: 推荐系统中的场 - 矩阵分解机

$FM^2$: Field-matrixed Factorization Machines for Recommender Systems

Counting the number of birds in an open sky setting has been an challenging
problem due to the large number of bird flocks and the birds can overlap.
Another difficulty is the lack of accurate training samples since the cost of
labeling images of bird flocks can be extremely high and each sample picture
can contain thousands of birds in a high resolution image. Inspired by recent
work on training with synthetic data to perform crowd counting, we design a
mechanism to generate synthetic bird dataset with precise bird count and the
corresponding density maps. We then train a Unet model on the synthetic dataset
to perform density map estimation that produces the count for each input. Our
method is able to achieve MSE of approximately 12.4 on real dataset. In order
to build a scalable system for fast bird counting under storage and
computational constraints, we use model compression techniques and efficient
model structures to increase the inference speed and save storage cost. We are
able to reduce storage cost from 55MB to less than 5MB for the model with
minimum loss of accuracy. This paper describes the pipelines of building an
efficient bird counting system.

本文介绍了建立高效鸟类计数系统的流程，利用合成数据集进行密度图估计，使用模型压缩技术和高效模型结构降低了存储成本从而提高了推理速度。