In this report, we present our champion solution for Ego4D EgoSchema
Challenge in CVPR 2024. To deeply integrate the powerful egocentric captioning
model and question reasoning model, we propose a novel Hierarchical
Comprehension scheme for egocentric video Question Answering, named HCQA. It
consists of three stages: Fine-grained Caption Generation, Context-driven
Summarization, and Inference-guided Answering. Given a long-form video, HCQA
captures local detailed visual information and global summarised visual
information via Fine-grained Caption Generation and Context-driven
Summarization, respectively. Then in Inference-guided Answering, HCQA utilizes
this hierarchical information to reason and answer given question. On the
EgoSchema blind test set, HCQA achieves 75% accuracy in answering over 5,000
human curated multiple-choice questions. Our code will be released at
this https URL

我们提出了一种新颖的分层理解方案 HCQA，用于自我中心视频问答，其中包括精细的字幕生成、上下文驱动的摘要生成和推理引导的回答。在 EgoSchema 盲测集中，HCQA 在回答超过 5000 个人工多项选择题中达到了 75% 的准确率。