The proliferation of deep learning-based machine vision applications has
given rise to a new type of compression, so called video coding for machine
(VCM). VCM differs from traditional video coding in that it is optimized for
machine vision performance instead of human visual quality. In the feature
compression track of MPEG-VCM, multi-scale features extracted from images are
subject to compression. Recent feature compression works have demonstrated that
the versatile video coding (VVC) standard-based approach can achieve a BD-rate
reduction of up to 96% against MPEG-VCM feature anchor. However, it is still
sub-optimal as VVC was not designed for extracted features but for natural
images. Moreover, the high encoding complexity of VVC makes it difficult to
design a lightweight encoder without sacrificing performance. To address these
challenges, we propose a novel multi-scale feature compression method that
enables both the end-to-end optimization on the extracted features and the
design of lightweight encoders. The proposed model combines a learnable
compressor with a multi-scale feature fusion network so that the redundancy in
the multi-scale features is effectively removed. Instead of simply cascading
the fusion network and the compression network, we integrate the fusion and
encoding processes in an interleaved way. Our model first encodes a
larger-scale feature to obtain a latent representation and then fuses the
latent with a smaller-scale feature. This process is successively performed
until the smallest-scale feature is fused and then the encoded latent at the
final stage is entropy-coded for transmission. The results show that our model
outperforms previous approaches by at least 52% BD-rate reduction and has
$\times5$ to $\times27$ times less encoding time for object detection. It is
noteworthy that our model can attain near-lossless task performance with only
0.002-0.003% of the uncompressed feature data size.

提出了一种新颖的多尺度特征压缩方法，并结合可学习的压缩器和多尺度特征融合网络实现端到端的优化，以及轻型编码器的设计。该方法在对象检测中取得了着眼的性能提升，并且在仅保留 0.002-0.003% 未压缩特征数据量的情况下，仍可以实现接近无损的性能。

端到端可学习的多尺度特征压缩技术适用于视觉编码器

End-to-End Learnable Multi-Scale Feature Compression for VCM

Semantic code search is the task of retrieving relevant code snippet given a
natural language query. Different from typical information retrieval tasks,
code search requires to bridge the semantic gap between the programming
language and natural language, for better describing intrinsic concepts and
semantics. Recently, deep neural network for code search has been a hot
research topic. Typical methods for neural code search first represent the code
snippet and query text as separate embeddings, and then use vector distance
(e.g. dot-product or cosine) to calculate the semantic similarity between them.
There exist many different ways for aggregating the variable length of code or
query tokens into a learnable embedding, including bi-encoder, cross-encoder,
and poly-encoder. The goal of the query encoder and code encoder is to produce
embeddings that are close with each other for a related pair of query and the
corresponding desired code snippet, in which the choice and design of encoder
is very significant.
In this paper, we propose a novel deep semantic model which makes use of the
utilities of not only the multi-modal sources, but also feature extractors such
as self-attention, the aggregated vectors, combination of the intermediate
representations. We apply the proposed model to tackle the CodeSearchNet
challenge about semantic code search. We align cross-lingual embedding for
multi-modality learning with large batches and hard example mining, and combine
different learned representations for better enhancing the representation
learning. Our model is trained on CodeSearchNet corpus and evaluated on the
held-out data, the final model achieves 0.384 NDCG and won the first place in
this benchmark. Models and code are available at
this https URL.

本文提出了一种新颖的深度语义模型，利用多模式来源和特征提取器的效用，将交叉语言嵌入与大批量和难样例挖掘结合起来，结合不同的学习表示来增强表示学习，以解决语义代码搜索任务，该模型在 CodeSearchNet 语料库上训练并在保留数据上评估，取得了 0.384 NDCG 的优异成绩，并在该基准测试中获得第一名。