Over 1.5 billion people worldwide live with hearing impairment. Despite
various technologies that have been created for individuals with such
disabilities, most of these technologies are either extremely expensive or
inaccessible for everyday use in low-medium income countries. In order to
combat this issue, we have developed a new assistive device, EchoVest, for
blind/deaf people to intuitively become more aware of their environment.
EchoVest transmits vibrations to the user's body by utilizing transcutaneous
electric nerve stimulation (TENS) based on the source of the sounds. EchoVest
also provides various features, including sound localization, sound
classification, noise reduction, and depth perception. We aimed to outperform
CNN-based machine-learning models, the most commonly used machine learning
model for classification tasks, in accuracy and computational costs. To do so,
we developed and employed a novel audio pipeline that adapts the Audio
Spectrogram Transformer (AST) model, an attention-based model, for our sound
classification purposes, and Fast Fourier Transforms for noise reduction. The
application of Otsu's Method helped us find the optimal thresholds for
background noise sound filtering and gave us much greater accuracy. In order to
calculate direction and depth accurately, we applied Complex Time Difference of
Arrival algorithms and SOTA localization. Our last improvement was to use blind
source separation to make our algorithms applicable to multiple microphone
inputs. The final algorithm achieved state-of-the-art results on numerous
checkpoints, including a 95.7\% accuracy on the ESC-50 dataset for
environmental sound classification.

我们开发了一种新的辅助设备 EchoVest，通过利用经皮电神经刺激 (TENS) 基于声源的振动传输到用户身体上，为盲人 / 聋人提供直观的环境感知，同时实现声音定位、声音分类、降噪和深度感知等多种功能，并在计算准确性和成本方面超越了基于 CNN 的机器学习模型。