Object-centric learning aims to break down complex visual scenes into more
manageable object representations, enhancing the understanding and reasoning
abilities of machine learning systems toward the physical world. Recently,
slot-based video models have demonstrated remarkable proficiency in segmenting
and tracking objects, but they overlook the importance of the effective
reasoning module. In the real world, reasoning and predictive abilities play a
crucial role in human perception and object tracking; in particular, these
abilities are closely related to human intuitive physics. Inspired by this, we
designed a novel reasoning module called the Slot-based Time-Space Transformer
with Memory buffer (STATM) to enhance the model's perception ability in complex
scenes. The memory buffer primarily serves as storage for slot information from
upstream modules, the Slot-based Time-Space Transformer makes predictions
through slot-based spatiotemporal attention computations and fusion. Our
experiment results on various datasets show that STATM can significantly
enhance object-centric learning capabilities of slot-based video models.

通过设计一种名为 STATM 的新型推理模块，可以显著增强基于插槽的视频模型的物体中心学习能力。

视频的推理增强的物体中心学习

Reasoning-Enhanced Object-Centric Learning for Videos

Despite significant success of deep learning in object detection tasks, the
standard training of deep neural networks requires access to a substantial
quantity of annotated images across all classes. Data annotation is an arduous
and time-consuming endeavor, particularly when dealing with infrequent objects.
Few-shot object detection (FSOD) methods have emerged as a solution to the
limitations of classic object detection approaches based on deep learning. FSOD
methods demonstrate remarkable performance by achieving robust object detection
using a significantly smaller amount of training data. A challenge for FSOD is
that instances from novel classes that do not belong to the fixed set of
training classes appear in the background and the base model may pick them up
as potential objects. These objects behave similarly to label noise because
they are classified as one of the training dataset classes, leading to FSOD
performance degradation. We develop a semi-supervised algorithm to detect and
then utilize these unlabeled novel objects as positive samples during the FSOD
training stage to improve FSOD performance. Specifically, we develop a
hierarchical ternary classification region proposal network (HTRPN) to localize
the potential unlabeled novel objects and assign them new objectness labels to
distinguish these objects from the base training dataset classes. Our improved
hierarchical sampling strategy for the region proposal network (RPN) also
boosts the perception ability of the object detection model for large objects.
We test our approach and COCO and PASCAL VOC baselines that are commonly used
in FSOD literature. Our experimental results indicate that our method is
effective and outperforms the existing state-of-the-art (SOTA) FSOD methods.
Our implementation is provided as a supplement to support reproducibility of
the results.

我们开发了一种半监督算法，通过检测和利用未标记的新颖对象来改善少样本目标检测的性能。我们的方法使用层次化三元分类算法以区分这些对象，并通过改善区域提议网络的感知能力来提高目标检测模型对大型对象的检测性能。实验结果表明，我们的方法是有效的，并且优于现有的少样本目标检测方法。

增强的少样本目标检测的改进区域建议网络

Improved Region Proposal Network for Enhanced Few-Shot Object Detection

Conventional training of deep neural networks requires a large number of the
annotated image which is a laborious and time-consuming task, particularly for
rare objects. Few-shot object detection (FSOD) methods offer a remedy by
realizing robust object detection using only a few training samples per class.
An unexplored challenge for FSOD is that instances from unlabeled novel classes
that do not belong to the fixed set of training classes appear in the
background. These objects behave similarly to label noise, leading to FSOD
performance degradation. We develop a semi-supervised algorithm to detect and
then utilize these unlabeled novel objects as positive samples during training
to improve FSOD performance. Specifically, we propose a hierarchical ternary
classification region proposal network (HTRPN) to localize the potential
unlabeled novel objects and assign them new objectness labels. Our improved
hierarchical sampling strategy for the region proposal network (RPN) also
boosts the perception ability of the object detection model for large objects.
Our experimental results indicate that our method is effective and outperforms
the existing state-of-the-art (SOTA) FSOD methods.

利用半监督学习技术，通过自动定位和使用未标记的新对象提高少样本目标检测的性能，并通过新的区域建议网络策略提高目标检测模型的感知能力，取得了超过现有最先进方法的性能。

提高少样本目标检测的新类别识别

Identification of Novel Classes for Improving Few-Shot Object Detection

Sight and hearing are two senses that play a vital role in human
communication and scene understanding. To mimic human perception ability,
audio-visual learning, aimed at developing computational approaches to learn
from both audio and visual modalities, has been a flourishing field in recent
years. A comprehensive survey that can systematically organize and analyze
studies of the audio-visual field is expected. Starting from the analysis of
audio-visual cognition foundations, we introduce several key findings that have
inspired our computational studies. Then, we systematically review the recent
audio-visual learning studies and divide them into three categories:
audio-visual boosting, cross-modal perception and audio-visual collaboration.
Through our analysis, we discover that, the consistency of audio-visual data
across semantic, spatial and temporal support the above studies. To revisit the
current development of the audio-visual learning field from a more macro view,
we further propose a new perspective on audio-visual scene understanding, then
discuss and analyze the feasible future direction of the audio-visual learning
area. Overall, this survey reviews and outlooks the current audio-visual
learning field from different aspects. We hope it can provide researchers with
a better understanding of this area. A website including constantly-updated
survey is released: https://gewu-lab.github.io/audio-visual-learning/.

介绍了视听感知、跨模态感知和协同感知在计算机视听学习中的重要性，综述了该领域的最新发展，并提出了关于场景理解的新视角和未来发展方向。