Exoskeletons for daily use by those with mobility impairments are being
developed. They will require accurate and robust scene understanding systems.
Current research has used vision to identify immediate terrain and geometric
obstacles, however these approaches are constrained to detections directly in
front of the user and are limited to classifying a finite range of terrain
types (e.g., stairs, ramps and level-ground). This paper presents Exosense, a
vision-centric scene understanding system which is capable of generating rich,
globally-consistent elevation maps, incorporating both semantic and terrain
traversability information. It features an elastic Atlas mapping framework
associated with a visual SLAM pose graph, embedded with open-vocabulary room
labels from a Vision-Language Model (VLM). The device's design includes a wide
field-of-view (FoV) fisheye multi-camera system to mitigate the challenges
introduced by the exoskeleton walking pattern. We demonstrate the system's
robustness to the challenges of typical periodic walking gaits, and its ability
to construct accurate semantically-rich maps in indoor settings. Additionally,
we showcase its potential for motion planning -- providing a step towards safe
navigation for exoskeletons.

通过视觉为核心的场景理解系统，本论文提出了 Exosense，它能够生成丰富的全局一致性高程图，包括语义和地形可通过性信息，并展示了其对于周期性步行步态的挑战的鲁棒性以及在室内环境中构建准确的语义丰富地图的能力，同时展示了其在运动规划方面的潜力，为外骨骼安全导航迈出了一步。

Exosense：适用于安全外骨骼导航的视觉中心场景理解系统

Exosense: A Vision-Centric Scene Understanding System For Safe  Exoskeleton Navigation

Human-robot walking with prosthetic legs and exoskeletons, especially over
complex terrains such as stairs, remains a significant challenge. Egocentric
vision has the unique potential to detect the walking environment prior to
physical interactions, which can improve transitions to and from stairs. This
motivated us to create the StairNet initiative to support the development of
new deep learning models for visual sensing and recognition of stairs, with an
emphasis on lightweight and efficient neural networks for onboard real-time
inference. In this study, we present an overview of the development of our
large-scale dataset with over 515,000 manually labeled images, as well as our
development of different deep learning models (e.g., 2D and 3D CNN, hybrid CNN
and LSTM, and ViT networks) and training methods (e.g., supervised learning
with temporal data and semi-supervised learning with unlabeled images) using
our new dataset. We consistently achieved high classification accuracy (i.e.,
up to 98.8%) with different designs, offering trade-offs between model accuracy
and size. When deployed on mobile devices with GPU and NPU accelerators, our
deep learning models achieved inference speeds up to 2.8 ms. We also deployed
our models on custom-designed CPU-powered smart glasses. However, limitations
in the embedded hardware yielded slower inference speeds of 1.5 seconds,
presenting a trade-off between human-centered design and performance. Overall,
we showed that StairNet can be an effective platform to develop and study new
visual perception systems for human-robot locomotion with applications in
exoskeleton and prosthetic leg control.

使用 StairNet 平台进行人机行走的研究，使用深度学习模型和数据集检测和识别楼梯，在有效模型精度和尺寸之间找到权衡，以提高复杂地面的行走效果。

StairNet: 用于人机移动的楼梯视觉识别

StairNet: Visual Recognition of Stairs for Human-Robot Locomotion

In this study, a gait phase classification method based on SVM multiclass
classification is introduced, with a focus on the precise identification of the
stance and swing phases, which are further subdivided into seven phases. Data
from individual IMU sensors, such as Shank Acceleration X, Y, Z, Shank Gyro X,
and Knee Angles, are used as features in this classification model. The
suggested technique successfully classifies the various gait phases with a
significant accuracy of about 90.3%. Gait phase classification is crucial,
especially in the domains of exoskeletons and prosthetics, where accurate
identification of gait phases enables seamless integration with assistive
equipment, improving mobility, stability, and energy economy. This study
extends the study of gait and offers an effective method for correctly
identifying gait phases from Shank IMU sensor data, with potential applications
in biomechanical research, exoskeletons, rehabilitation, and prosthetics.

基于 SVM 多类分类的步态相位分类方法准确识别七个相位，使用 IMU 传感器数据作为特征。该技术可成功分类不同步态相位，准确度达到 90.3%，对于外骨骼和假肢领域至关重要，可实现与辅助装置的无缝集成，提高行动能力、稳定性和能量效益。该研究扩展了步态学研究，提供了一种有效的方法，能从 Shank IMU 传感器数据中准确识别步态相位，并在生物力学研究、外骨骼、康复和假肢等领域有潜在应用。