We propose a novel method to estimate a driver's points-of-gaze using a pair
of ordinary cameras mounted on the windshield and dashboard of a car. This is a
challenging problem due to the dynamics of traffic environments with 3D scenes
of unknown depths. This problem is further complicated by the volatile distance
between the driver and the camera system. To tackle these challenges, we
develop a novel convolutional network that simultaneously analyzes the image of
the scene and the image of the driver's face. This network has a camera
calibration module that can compute an embedding vector that represents the
spatial configuration between the driver and the camera system. This
calibration module improves the overall network's performance, which can be
jointly trained end to end.
We also address the lack of annotated data for training and evaluation by
introducing a large-scale driving dataset with point-of-gaze annotations. This
is an in situ dataset of real driving sessions in an urban city, containing
synchronized images of the driving scene as well as the face and gaze of the
driver. Experiments on this dataset show that the proposed method outperforms
various baseline methods, having the mean prediction error of 29.69 pixels,
which is relatively small compared to the $1280{\times}720$ resolution of the
scene camera.

我们提出了一种使用安装在车辆挡风玻璃和仪表盘上的普通摄像头来估计驾驶员凝视点的新方法。该方法通过开发一种卷积网络来同时分析场景图像和驾驶员面部图像，其中包含一个相机校准模块，可以计算表示驾驶员和摄像头系统之间空间配置的嵌入向量，该模块改善了网络的性能并可以进行端到端的联合训练。我们还引入了一种大规模的驾驶数据集，其中包含了真实驾驶情景的图像以及驾驶员面部和凝视数据，对该数据集的实验表明，所提出的方法优于各种基准方法，均方误差为 29.69 像素，相对于场景摄像头的 $1280 {imes} 720$ 分辨率而言，误差较小。

驾驶员注意力跟踪和分析

Driver Attention Tracking and Analysis

Few-shot object detection, which focuses on detecting novel objects with few
labels, is an emerging challenge in the community. Recent studies show that
adapting a pre-trained model or modified loss function can improve performance.
In this paper, we explore leveraging the power of Contrastive Language-Image
Pre-training (CLIP) and hard negative classification loss in low data setting.
Specifically, we propose Re-scoring using Image-language Similarity for
Few-shot object detection (RISF) which extends Faster R-CNN by introducing
Calibration Module using CLIP (CM-CLIP) and Background Negative Re-scale Loss
(BNRL). The former adapts CLIP, which performs zero-shot classification, to
re-score the classification scores of a detector using image-class
similarities, the latter is modified classification loss considering the
punishment for fake backgrounds as well as confusing categories on a
generalized few-shot object detection dataset. Extensive experiments on MS-COCO
and PASCAL VOC show that the proposed RISF substantially outperforms the
state-of-the-art approaches. The code will be available.

利用 Contrastive Language-Image Pre-training (CLIP) 和 hard negative classification loss，在少数据环境下通过 Re-scoring using Image-language Similarity for Few-shot object detection (RISF) 方法提升了 few-shot 目标检测性能。

借助图像 - 语言相似性的重新打分方法用于小样本目标检测

Re-Scoring Using Image-Language Similarity for Few-Shot Object Detection

New classes arise frequently in our ever-changing world, e.g., emerging
topics in social media and new types of products in e-commerce. A model should
recognize new classes and meanwhile maintain discriminability over old classes.
Under severe circumstances, only limited novel instances are available to
incrementally update the model. The task of recognizing few-shot new classes
without forgetting old classes is called few-shot class-incremental learning
(FSCIL). In this work, we propose a new paradigm for FSCIL based on
meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT), which
synthesizes fake FSCIL tasks from the base dataset. The data format of fake
tasks is consistent with the `real' incremental tasks, and we can build a
generalizable feature space for the unseen tasks through meta-learning.
Besides, LIMIT also constructs a calibration module based on transformer, which
calibrates the old class classifiers and new class prototypes into the same
scale and fills in the semantic gap. The calibration module also adaptively
contextualizes the instance-specific embedding with a set-to-set function.
LIMIT efficiently adapts to new classes and meanwhile resists forgetting over
old classes. Experiments on three benchmark datasets (CIFAR100, miniImageNet,
and CUB200) and large-scale dataset, i.e., ImageNet ILSVRC2012 validate that
LIMIT achieves state-of-the-art performance.

本研究提出基于元学习的 LearnIng Multi-phase Incremental Tasks (LIMIT) 方法，合成假的 Few-shot Class-Incremental Learning 任务来建立分类特征空间，并基于 Transformer 构建校准模块，实现新类别的适应和对旧类别的防遗忘，该方法在多个数据集上实现了最好表现。

通过抽样多相任务进行少样本类增量学习

Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks

A key solution to temporal sentence grounding (TSG) exists in how to learn
effective alignment between vision and language features extracted from an
untrimmed video and a sentence description. Existing methods mainly leverage
vanilla soft attention to perform the alignment in a single-step process.
However, such single-step attention is insufficient in practice, since
complicated relations between inter- and intra-modality are usually obtained
through multi-step reasoning. In this paper, we propose an Iterative Alignment
Network (IA-Net) for TSG task, which iteratively interacts inter- and
intra-modal features within multiple steps for more accurate grounding.
Specifically, during the iterative reasoning process, we pad multi-modal
features with learnable parameters to alleviate the nowhere-to-attend problem
of non-matched frame-word pairs, and enhance the basic co-attention mechanism
in a parallel manner. To further calibrate the misaligned attention caused by
each reasoning step, we also devise a calibration module following each
attention module to refine the alignment knowledge. With such iterative
alignment scheme, our IA-Net can robustly capture the fine-grained relations
between vision and language domains step-by-step for progressively reasoning
the temporal boundaries. Extensive experiments conducted on three challenging
benchmarks demonstrate that our proposed model performs better than the
state-of-the-arts.

本文提出了一个迭代对齐网络 (IA-Net) 来解决 TSG 任务，通过多步推理来互动地对齐视觉和语言特征，通过学习参数对多模态特征进行填充来缓解非匹配的问题，并且在并行方式下增强基本的共同注意力机制。为了进一步校准由每个推理步骤引起的不匹配的注意力，我们还设计了一个校准模块。本文的 IA-Net 模型在三个具有挑战性的基准测试中表现出更好的性能。