In recent years, as machine learning, particularly for vision and language
understanding, has been improved, research in embedded AI has also evolved.
VOYAGER is a well-known LLM-based embodied AI that enables autonomous
exploration in the Minecraft world, but it has issues such as underutilization
of visual data and insufficient functionality as a world model. In this
research, the possibility of utilizing visual data and the function of LLM as a
world model were investigated with the aim of improving the performance of
embodied AI. The experimental results revealed that LLM can extract necessary
information from visual data, and the utilization of the information improves
its performance as a world model. It was also suggested that devised prompts
could bring out the LLM's function as a world model.

在这项研究中，通过探索利用视觉数据和 LLM 作为世界模型的可能性，以改善嵌入式人工智能的性能，实验结果表明 LLM 可以从视觉数据中提取必要信息，并利用这些信息提高其作为世界模型的性能，同时还指出设计的提示可以发挥 LLM 作为世界模型的功能。

基于 LLM 的具有视觉信息和面向预测的具身化世界模型

The Embodied World Model Based on LLM with Visual Information and  Prediction-Oriented Prompts

The rise of power-efficient embedded computers based on highly-parallel
accelerators opens a number of opportunities and challenges for researchers and
engineers, and paved the way to the era of edge computing. At the same time,
advances in embedded AI for object detection and categorization such as YOLO,
GoogleNet and AlexNet reached an unprecedented level of accuracy (mean-Average
Precision - mAP) and performance (Frames-Per-Second - FPS). Today, edge
computers based on heterogeneous many-core systems are a predominant choice to
deploy such systems in industry 4.0, wearable devices, and - our focus -
autonomous driving systems. In these latter systems, engineers struggle to make
reduced automotive power and size budgets co-exist with the accuracy and
performance targets requested by autonomous driving. We aim at validating the
effectiveness and efficiency of most recent networks on state-of-the-art
platforms with embedded commercial-off-the-shelf System-on-Chips, such as
Xavier AGX, Tegra X2 and Nano for NVIDIA and XCZU9EG and XCZU3EG of the Zynq
UltraScale+ family, for the Xilinx counterpart. Our work aims at supporting
engineers in choosing the most appropriate CNN package and computing system for
their designs, and deriving guidelines for adequately sizing their systems.

最近的网络在嵌入式商用系统芯片上的有效性和效率进行了验证，为工程师选择最合适的卷积神经网络包和计算系统提供了指导。

嵌入式汽车平台上卷积神经网络包的性能 / 功耗评估

Performance/power assessment of CNN packages on embedded automotive  platforms

This paper introduces a new lightweight method for image recognition.
ImageSig is based on computing signatures and does not require a convolutional
structure or an attention-based encoder. It is striking to the authors that it
achieves: a) an accuracy for 64 X 64 RGB images that exceeds many of the
state-of-the-art methods and simultaneously b) requires orders of magnitude
less FLOPS, power and memory footprint. The pretrained model can be as small as
44.2 KB in size. ImageSig shows unprecedented performance on hardware such as
Raspberry Pi and Jetson-nano. ImageSig treats images as streams with multiple
channels. These streams are parameterized by spatial directions. We contribute
to the functionality of signature and rough path theory to stream-like data and
vision tasks on static images beyond temporal streams. With very few parameters
and small size models, the key advantage is that one could have many of these
"detectors" assembled on the same chip; moreover, the feature acquisition can
be performed once and shared between different models of different tasks -
further accelerating the process. This contributes to energy efficiency and the
advancements of embedded AI at the edge.

这篇论文介绍了一种基于 Signature computation 的轻量级图像识别方法，它能在 64X64 的 RGB 图像上达到州界方法的准确率，但所需的 FLOPS、计算资源和内存占用都远少于其它方法，预训练模型的大小可以减少到 44.2KB，该方法可以加速嵌入式 AI 的发展和应用。