Audio commands are a preferred communication medium to keep inspectors in the
loop of civil infrastructure inspection performed by a semi-autonomous drone.
To understand job-specific commands from a group of heterogeneous and dynamic
inspectors, a model must be developed cost-effectively for the group and easily
adapted when the group changes. This paper is motivated to build a
multi-tasking deep learning model that possesses a Share-Split-Collaborate
architecture. This architecture allows the two classification tasks to share
the feature extractor and then split subject-specific and keyword-specific
features intertwined in the extracted features through feature projection and
collaborative training. A base model for a group of five authorized subjects is
trained and tested on the inspection keyword dataset collected by this study.
The model achieved a 95.3% or higher mean accuracy in classifying the keywords
of any authorized inspectors. Its mean accuracy in speaker classification is
99.2%. Due to the richer keyword representations that the model learns from the
pooled training data, adapting the base model to a new inspector requires only
a little training data from that inspector, like five utterances per keyword.
Using the speaker classification scores for inspector verification can achieve
a success rate of at least 93.9% in verifying authorized inspectors and 76.1%
in detecting unauthorized ones. Further, the paper demonstrates the
applicability of the proposed model to larger-size groups on a public dataset.
This paper provides a solution to addressing challenges facing AI-assisted
human-robot interaction, including worker heterogeneity, worker dynamics, and
job heterogeneity.

该研究旨在构建一种具有共享分裂协作结构的多任务深度学习模型，用于通过音频命令，对半自主无人机进行民用基础设施检查，该模型可以分类任何授权检查员的关键词，并在说话者分类方面取得了 99.2％的平均准确度。同时，该论文还探讨了该模型在人机交互中的应用价值。