We present a transfer learning analysis on a sporting environment of the
expanded 3D (X3D) neural networks. Inspired by action quality assessment
methods in the literature, our method uses an action recognition network to
estimate athletes' cumulative race time (CRT) during an ultra-distance
competition. We evaluate the performance considering the X3D, a family of
action recognition networks that expand a small 2D image classification
architecture along multiple network axes, including space, time, width, and
depth. We demonstrate that the resulting neural network can provide remarkable
performance for short input footage, with a mean absolute error of 12 minutes
and a half when estimating the CRT for runners who have been active from 8 to
20 hours. Our most significant discovery is that X3D achieves state-of-the-art
performance while requiring almost seven times less memory to achieve better
precision than previous work.

我们对扩展 3D（X3D）神经网络在体育环境中进行了迁移学习分析，通过运动识别网络来估计运动员在超长距离比赛期间的累计比赛时间（CRT）。我们评估了 X3D 的性能，这是一种扩展小型 2D 图像分类体系结构的行动识别网络，可以沿多个网络轴（包括空间、时间、宽度和深度）扩展。我们证明了该神经网络在短时间内可以提供出色的性能，对于活动时间在 8 到 20 个小时之间的跑步者估计 CRT 的平均绝对误差仅为 12 分钟半。我们最重要的发现是，相较于以前的工作，X3D 在达到更好的精度时所需的内存几乎要少七倍。

一种用于野外运动环境中跑者性能评估的 X3D 神经网络分析

An X3D Neural Network Analysis for Runner's Performance Assessment in a  Wild Sporting Environment

3D neural networks are widely used in real-world applications (e.g., AR/VR
headsets, self-driving cars). They are required to be fast and accurate;
however, limited hardware resources on edge devices make these requirements
rather challenging. Previous work processes 3D data using either voxel-based or
point-based neural networks, but both types of 3D models are not
hardware-efficient due to the large memory footprint and random memory access.
In this paper, we study 3D deep learning from the efficiency perspective. We
first systematically analyze the bottlenecks of previous 3D methods. We then
combine the best from point-based and voxel-based models together and propose a
novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We
further enhance this primitive with the sparse convolution to make it more
effective in processing large (outdoor) scenes. Based on our designed 3D
primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the
best 3D network architecture given a resource constraint. We evaluate our
proposed method on six representative benchmark datasets, achieving
state-of-the-art performance with 1.8-23.7x measured speedup. Furthermore, our
method has been deployed to the autonomous racing vehicle of MIT Driverless,
achieving larger detection range, higher accuracy and lower latency.

该研究通过结合基于点和基于体素的模型，提出一种硬件高效的三维卷积方法，并进一步融合 稀疏卷积，以更有效地处理大场景，从硬件效率的角度系统研究三维深度学习。在此基础上，引入三维神经架构搜索来探索最佳三维网络架构，并在六个典型基准数据集上评估其性能，实现 1.8-23.7 倍的加速，已应用于 MIT Driverless 无人驾驶车辆上，实现了更大的检测范围，更高的准确性和更低的延迟。

PVNAS: 3D 点 - 体素卷积神经结构搜索

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution

Most 3D neural networks are trained from scratch owing to the lack of
large-scale labeled 3D datasets. In this paper, we present a novel 3D
pretraining method by leveraging 2D networks learned from rich 2D datasets. We
propose the contrastive pixel-to-point knowledge transfer to effectively
utilize the 2D information by mapping the pixel-level and point-level features
into the same embedding space. Due to the heterogeneous nature between 2D and
3D networks, we introduce the back-projection function to align the features
between 2D and 3D to make the transfer possible. Additionally, we devise an
upsampling feature projection layer to increase the spatial resolution of
high-level 2D feature maps, which enables learning fine-grained 3D
representations. With a pretrained 2D network, the proposed pretraining process
requires no additional 2D or 3D labeled data, further alleviating the expensive
3D data annotation cost. To the best of our knowledge, we are the first to
exploit existing 2D trained weights to pretrain 3D deep neural networks. Our
intensive experiments show that the 3D models pretrained with 2D knowledge
boost the performances of 3D networks across various real-world 3D downstream
tasks.

本文提出了一种使用 2D 网络预训练 3D 深度神经网络的新方法，包括像素到点知识转移、反向投影函数、上采样特征投影层等，成功提升了各种真实世界 3D 网络下游任务的模型性能。