Few-shot object detection~(FSOD), which aims to detect novel objects with
limited annotated instances, has made significant progress in recent years.
However, existing methods still suffer from biased representations, especially
for novel classes in extremely low-shot scenarios. During fine-tuning, a novel
class may exploit knowledge from similar base classes to construct its own
feature distribution, leading to classification confusion and performance
degradation. To address these challenges, we propose a fine-tuning based FSOD
framework that utilizes semantic embeddings for better detection. In our
proposed method, we align the visual features with class name embeddings and
replace the linear classifier with our semantic similarity classifier. Our
method trains each region proposal to converge to the corresponding class
embedding. Furthermore, we introduce a multimodal feature fusion to augment the
vision-language communication, enabling a novel class to draw support
explicitly from well-trained similar base classes. To prevent class confusion,
we propose a semantic-aware max-margin loss, which adaptively applies a margin
beyond similar classes. As a result, our method allows each novel class to
construct a compact feature space without being confused with similar base
classes. Extensive experiments on Pascal VOC and MS COCO demonstrate the
superiority of our method.

提出了一个使用语义嵌入进行精细调整的少样本目标检测框架，通过使用语义相似性分类器，多模态特征融合和语义感知最大边界损失，解决了现有方法在极低样本场景中对新类别存在的分类混淆和性能下降问题。

语义增强的少样本目标检测

Semantic Enhanced Few-shot Object Detection

Talking face synthesis driven by audio is one of the current research
hotspots in the fields of multidimensional signal processing and multimedia.
Neural Radiance Field (NeRF) has recently been brought to this research field
in order to enhance the realism and 3D effect of the generated faces. However,
most existing NeRF-based methods either burden NeRF with complex learning tasks
while lacking methods for supervised multimodal feature fusion, or cannot
precisely map audio to the facial region related to speech movements. These
reasons ultimately result in existing methods generating inaccurate lip shapes.
This paper moves a portion of NeRF learning tasks ahead and proposes a talking
face synthesis method via NeRF with attention-based disentanglement (NeRF-AD).
In particular, an Attention-based Disentanglement module is introduced to
disentangle the face into Audio-face and Identity-face using speech-related
facial action unit (AU) information. To precisely regulate how audio affects
the talking face, we only fuse the Audio-face with audio feature. In addition,
AU information is also utilized to supervise the fusion of these two
modalities. Extensive qualitative and quantitative experiments demonstrate that
our NeRF-AD outperforms state-of-the-art methods in generating realistic
talking face videos, including image quality and lip synchronization. To view
video results, please refer to this https URL

通过引入基于注意力的解缠模块（NeRF-AD），本文提出了一种基于 Neural Radiance Field 的语音驱动说话人脸合成方法，可以生成逼真且具有嘴唇同步效果的说话人脸视频。

NeRF-AD: 基于注意力解耦的神经辐射场对话人脸合成

NeRF-AD: Neural Radiance Field with Attention-based Disentanglement for  Talking Face Synthesis

With short video platforms becoming one of the important channels for news
sharing, major short video platforms in China have gradually become new
breeding grounds for fake news. However, it is not easy to distinguish short
video rumors due to the great amount of information and features contained in
short videos, as well as the serious homogenization and similarity of features
among videos. In order to mitigate the spread of short video rumors, our group
decides to detect short video rumors by constructing multimodal feature fusion
and introducing external knowledge after considering the advantages and
disadvantages of each algorithm. The ideas of detection are as follows: (1)
dataset creation: to build a short video dataset with multiple features; (2)
multimodal rumor detection model: firstly, we use TSN (Temporal Segment
Networks) video coding model to extract video features; then, we use OCR
(Optical Character Recognition) and ASR (Automatic Character Recognition) to
extract video features. Recognition) and ASR (Automatic Speech Recognition)
fusion to extract text, and then use the BERT model to fuse text features with
video features (3) Finally, use contrast learning to achieve distinction: first
crawl external knowledge, then use the vector database to achieve the
introduction of external knowledge and the final structure of the
classification output. Our research process is always oriented to practical
needs, and the related knowledge results will play an important role in many
practical scenarios such as short video rumor identification and social opinion
control.

该研究探讨了利用多模式特征融合和引入外部知识来检测短视频假消息的方法，包括数据集构建，多模式谣言检测模型和对比学习技术，并在实际应用场景中发挥作用。