We present a parameter-efficient method for continual video
question-answering (VidQA) learning. Our method, named DAM, uses the proposed
Dynamic Adapter Merging to (i) mitigate catastrophic forgetting, (ii) enable
efficient adaptation to continually arriving datasets, (iii) handle inputs from
unknown datasets during inference, and (iv) enable knowledge sharing across
similar dataset domains. Given a set of continually streaming VidQA datasets,
we sequentially train dataset-specific adapters for each dataset while freezing
the parameters of a large pretrained video-language backbone. During inference,
given a video-question sample from an unknown domain, our method first uses the
proposed non-parametric router function to compute a probability for each
adapter, reflecting how relevant that adapter is to the current video-question
input instance. Subsequently, the proposed dynamic adapter merging scheme
aggregates all the adapter weights into a new adapter instance tailored for
that particular test sample to compute the final VidQA prediction, mitigating
the impact of inaccurate router predictions and facilitating knowledge sharing
across domains. Our DAM model outperforms prior state-of-the-art continual
learning approaches by 9.1% while exhibiting 1.9% less forgetting on 6 VidQA
datasets spanning various domains. We further extend DAM to continual image
classification and image QA and outperform prior methods by a large margin. The
code is publicly available at: this https URL

我们提出了一种参数高效的方法用于连续视频问答（VidQA）学习，名为 DAM。我们的方法通过动态适配器合并技术来减轻灾难性遗忘、实现对不断到达的数据集的高效适应、处理推理过程中来自未知数据集的输入，并在相似数据集领域中实现知识共享。