Capturing the abnormal event from surveillance videos enhances the safety and
well-being of the citizens. The application of EdgeAI (Edge computing-based
Artificial Intelligent ) meets the strict latency requirements for security. In
this paper, we apply weakly supervised video anomaly detection called Robust
Temporal Feature Magnitude Learning (RTFM) to an end-to-end crime-scene anomaly
detection system from the surveillance cameras with the help of edge computing
technology. The system is tested directly on multiple Jetson edge devices
combined with TensorRT as the software developer kit from NVIDIA for system
performance enhancement. The experience of an AI-based system deployment on
various Jetson Edge devices with Docker technology is also provided. The
anomaly detection model yields competitive results compared to other
state-of-the-art (SOTA) algorithms on available datasets such as UCF-Crime and
UIT VNAnomaly. The approach system reaches 47.56 frames per second (FPS)
inference speed on a Jetson edge device with only 3.11 GB RAM usage total. We
also discover the promising Jetson device that the AI system achieves 15%
better performance than the previous version of Jetson devices while consuming
50% less energy power.

捕捉来自监控视频的异常事件提高了市民的安全和福祉。本文应用基于边缘计算的人工智能（EdgeAI）满足安全需求的低延迟要求。我们使用弱监督视频异常检测技术（RTFM）将端到端犯罪现场异常检测系统应用于监控摄像头，并借助边缘计算技术。该系统直接在多个 Jetson 边缘设备上进行测试，结合 NVIDIA 的软件开发工具包 TensorRT 以提高系统性能。该方法在可用数据集（如 UCF-Crime 和 UIT VNAnomaly）上与其他最先进算法相比取得了竞争力。该系统在使用仅 3.11GB RAM 的 Jetson 边缘设备上实现了每秒 47.56 帧的推理速度。我们还发现，该 AI 系统在能耗方面比早期版本的 Jetson 设备性能提高了 15%，而能耗降低了 50%。

在不同的 Jetson 边缘设备上对异常检测系统进行基准测试

Benchmarking Anomaly Detection System on various Jetson Edge Devices

Transformers' compute-intensive operations pose enormous challenges for their
deployment in resource-constrained EdgeAI / tinyML devices. As an established
neural network compression technique, quantization reduces the hardware
computational and memory resources. In particular, fixed-point quantization is
desirable to ease the computations using lightweight blocks, like adders and
multipliers, of the underlying hardware. However, deploying fully-quantized
Transformers on existing general-purpose hardware, generic AI accelerators, or
specialized architectures for Transformers with floating-point units might be
infeasible and/or inefficient.
Towards this, we propose SwiftTron, an efficient specialized hardware
accelerator designed for Quantized Transformers. SwiftTron supports the
execution of different types of Transformers' operations (like Attention,
Softmax, GELU, and Layer Normalization) and accounts for diverse scaling
factors to perform correct computations. We synthesize the complete SwiftTron
architecture in a $65$ nm CMOS technology with the ASIC design flow. Our
Accelerator executes the RoBERTa-base model in 1.83 ns, while consuming 33.64
mW power, and occupying an area of 273 mm^2. To ease the reproducibility, the
RTL of our SwiftTron architecture is released at
this https URL

本文提出了 SwiftTron，这是一种高效的专用硬件加速器，旨在支持量化 Transformer 的不同类型操作，并考虑了各种缩放因子以进行正确的计算，在 65 纳米 CMOS 技术中，RoBERTa-base 模型的加速器在 1.83 纳秒内执行，同时消耗 33.64 mW 的功率，占用 273 平方毫米的面积，可以缓解 Transformers 在资源受限的 EdgeAI /tinyML 设备上部署的挑战。

SwiftTron: 用于量化 Transformer 的高效硬件加速器

SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers

Despite its importance for federated learning, continuous learning and many
other applications, on-device training remains an open problem for EdgeAI. The
problem stems from the large number of operations (e.g., floating point
multiplications and additions) and memory consumption required during training
by the back-propagation algorithm. Consequently, in this paper, we propose a
new gradient filtering approach which enables on-device DNN model training.
More precisely, our approach creates a special structure with fewer unique
elements in the gradient map, thus significantly reducing the computational
complexity and memory consumption of back propagation during training.
Extensive experiments on image classification and semantic segmentation with
multiple DNN models (e.g., MobileNet, DeepLabV3, UPerNet) and devices (e.g.,
Raspberry Pi and Jetson Nano) demonstrate the effectiveness and wide
applicability of our approach. For example, compared to SOTA, we achieve up to
19$\times$ speedup and 77.1% memory savings on ImageNet classification with
only 0.1% accuracy loss. Finally, our method is easy to implement and deploy;
over 20$\times$ speedup and 90% energy savings have been observed compared to
highly optimized baselines in MKLDNN and CUDNN on NVIDIA Jetson Nano.
Consequently, our approach opens up a new direction of research with a huge
potential for on-device training.

本文提出了一种新的梯度过滤方法，使用少量的独特元素创建特殊的结构来显著减少反向传播期间的计算复杂度和内存消耗，从而实现了设备端深度学习模型训练，显著提高了速度和节省了能源。