This work addresses the problems of (a) designing utilization measurements of
trained artificial intelligence (AI) models and (b) explaining how training
data are encoded in AI models based on those measurements. The problems are
motivated by the lack of explainability of AI models in security and safety
critical applications, such as the use of AI models for classification of
traffic signs in self-driving cars. We approach the problems by introducing
theoretical underpinnings of AI model utilization measurement and understanding
patterns in utilization-based class encodings of traffic signs at the level of
computation graphs (AI models), subgraphs, and graph nodes. Conceptually,
utilization is defined at each graph node (computation unit) of an AI model
based on the number and distribution of unique outputs in the space of all
possible outputs (tensor-states). In this work, utilization measurements are
extracted from AI models, which include poisoned and clean AI models. In
contrast to clean AI models, the poisoned AI models were trained with traffic
sign images containing systematic, physically realizable, traffic sign
modifications (i.e., triggers) to change a correct class label to another label
in a presence of such a trigger. We analyze class encodings of such clean and
poisoned AI models, and conclude with implications for trojan injection and
detection.

研究了利用神经网络对交通标志进行分类的应用场景中人工智能模型的可解释性问题，提出了机器学习模型的利用率量化方法，并将其用于污染和清洁的人工智能模型中进行分析，讨论了木马注入和检测的影响。