This study aims to explore efficient tuning methods for the screenshot
captioning task. Recently, image captioning has seen significant advancements,
but research in captioning tasks for mobile screens remains relatively scarce.
Current datasets and use cases describing user behaviors within product
screenshots are notably limited. Consequently, we sought to fine-tune
pre-existing models for the screenshot captioning task. However, fine-tuning
large pre-trained models can be resource-intensive, requiring considerable
time, computational power, and storage due to the vast number of parameters in
image captioning models. To tackle this challenge, this study proposes a
combination of adapter methods, which necessitates tuning only the additional
modules on the model. These methods are originally designed for vision or
language tasks, and our intention is to apply them to address similar
challenges in screenshot captioning. By freezing the parameters of the image
caption models and training only the weights associated with the methods,
performance comparable to fine-tuning the entire model can be achieved, while
significantly reducing the number of parameters. This study represents the
first comprehensive investigation into the effectiveness of combining adapters
within the context of the screenshot captioning task. Through our experiments
and analyses, this study aims to provide valuable insights into the application
of adapters in vision-language models and contribute to the development of
efficient tuning techniques for the screenshot captioning task. Our study is
available at this https URL

本研究旨在探讨对屏幕截图字幕任务进行高效调优的方法。通过冻结图像字幕模型的参数，仅训练与方法相关的权重，可以实现与整个模型微调相当的性能，同时大幅减少参数数量。该研究首次全面探讨了在屏幕截图字幕任务中结合适配器的有效性，并通过实验和分析为适配器在视觉语言模型中的应用提供有价值的观点，为屏幕截图字幕任务的高效调优技术的发展作出贡献。