While neural networks have excelled in video action recognition tasks, their
black-box nature often obscures the understanding of their decision-making
processes. Recent approaches used inherently interpretable models to analyze
video actions in a manner akin to human reasoning. These models, however,
usually fall short in performance compared to their black-box counterparts. In
this work, we present a new framework named Language-guided Interpretable
Action Recognition framework (LaIAR). LaIAR leverages knowledge from language
models to enhance both the recognition capabilities and the interpretability of
video models. In essence, we redefine the problem of understanding video model
decisions as a task of aligning video and language models. Using the logical
reasoning captured by the language model, we steer the training of the video
model. This integrated approach not only improves the video model's
adaptability to different domains but also boosts its overall performance.
Extensive experiments on two complex video action datasets, Charades & CAD-120,
validates the improved performance and interpretability of our LaIAR framework.
The code of LaIAR is available at this https URL

通过语言模型引导的可解释的动作识别框架 (LaIAR)，提高了视频模型的性能和可解释性。