This paper introduces our solution for Track 2 in AI City Challenge 2024. The
task aims to solve traffic safety description and analysis with the dataset of
Woven Traffic Safety (WTS), a real-world Pedestrian-Centric Traffic Video
Dataset for Fine-grained Spatial-Temporal Understanding. Our solution mainly
focuses on the following points: 1) To solve dense video captioning, we
leverage the framework of dense video captioning with parallel decoding (PDVC)
to model visual-language sequences and generate dense caption by chapters for
video. 2) Our work leverages CLIP to extract visual features to more
efficiently perform cross-modality training between visual and textual
representations. 3) We conduct domain-specific model adaptation to mitigate
domain shift problem that poses recognition challenge in video understanding.
4) Moreover, we leverage BDD-5K captioned videos to conduct knowledge transfer
for better understanding WTS videos and more accurate captioning. Our solution
has yielded on the test set, achieving 6th place in the competition. The open
source code will be available at this https URL

该研究介绍了在 AI City Challenge 2024 中针对 Track 2 的解决方案，主要关注交通安全、视频字幕、跨模态训练和领域特定模型适应等关键词。该解决方案在竞赛中取得了第六名，并通过提供开源代码使其更广泛地应用。

利用并行密集视频字幕提高交通安全的端到端事件分析

Enhancing Traffic Safety with Parallel Dense Video Captioning for  End-to-End Event Analysis

This paper introduces a novel human pose estimation benchmark, Human Pose
with Millimeter Wave Radar (HuPR), that includes synchronized vision and radio
signal components. This dataset is created using cross-calibrated mmWave radar
sensors and a monocular RGB camera for cross-modality training of radar-based
human pose estimation. There are two advantages of using mmWave radar to
perform human pose estimation. First, it is robust to dark and low-light
conditions. Second, it is not visually perceivable by humans and thus, can be
widely applied to applications with privacy concerns, e.g., surveillance
systems in patient rooms. In addition to the benchmark, we propose a
cross-modality training framework that leverages the ground-truth 2D keypoints
representing human body joints for training, which are systematically generated
from the pre-trained 2D pose estimation network based on a monocular camera
input image, avoiding laborious manual label annotation efforts. The framework
consists of a new radar pre-processing method that better extracts the velocity
information from radar data, Cross- and Self-Attention Module (CSAM), to fuse
multi-scale radar features, and Pose Refinement Graph Convolutional Networks
(PRGCN), to refine the predicted keypoint confidence heatmaps. Our intensive
experiments on the HuPR benchmark show that the proposed scheme achieves better
human pose estimation performance with only radar data, as compared to
traditional pre-processing solutions and previous radio-frequency-based
methods.

本文提出了一种新型的人体姿态估计基准测试，使用交叉校准的毫米波雷达传感器和单眼 RGB 相机进行跨模态训练，在不涉及人类隐私的情况下，利用毫米波雷达的优势，通过 2D 关键点表示人体关节进行训练，并采用新的雷达预处理方法和姿态修正图卷积网络进行预测关键点的信心热图的优化，实现了比传统方法更好的人体姿态估计性能。