Language has become a prominent modality in computer vision with the rise of
multi-modal LLMs. Despite supporting long context-lengths, their effectiveness
in handling long-term information gradually declines with input length. This
becomes critical, especially in applications such as long-form video
understanding. In this paper, we introduce a Language Repository (LangRepo) for
LLMs, that maintains concise and structured information as an interpretable
(i.e., all-textual) representation. Our repository is updated iteratively based
on multi-scale video chunks. We introduce write and read operations that focus
on pruning redundancies in text, and extracting information at various temporal
scales. The proposed framework is evaluated on zero-shot visual
question-answering benchmarks including EgoSchema, NExT-QA, IntentQA and
NExT-GQA, showing state-of-the-art performance at its scale. Our code is
available at this https URL

本文介绍了一种语言仓库 (LangRepo)，用于长文本视频理解，通过维护简明有结构信息作为可解释的（即全文本）表示。该仓库基于多尺度视频块进行迭代更新，并引入了专注于修剪文本中的冗余信息和在各个时间尺度提取信息的写入和读取操作。该提出的框架在包括 EgoSchema、NExT-QA、IntentQA 和 NExT-GQA 等无监督视觉问答基准测试上进行评估，表现出最新最好的性能。我们的代码可在此链接获得：https:// 此链接。

长视频理解语言库

Language Repository for Long Video Understanding

This article presents our generative model for rhythm action games together
with applications in business operations. Rhythm action games are video games
in which the player is challenged to issue commands at the right timings during
a music session. The timings are rendered in the chart, which consists of
visual symbols, called notes, flying through the screen. We introduce our deep
generative model, Gen\'eLive!, which outperforms the state-of-the-art model by
taking into account musical structures through beats and temporal scales.
Thanks to its favorable performance, Gen\'eLive! was put into operation at KLab
Inc., a Japan-based video game developer, and reduced the business cost of
chart generation by as much as half. The application target included the
phenomenal "Love Live!," which has more than 10 million users across Asia and
beyond, and is one of the few rhythm action franchises that has led the online
era of the genre. In this article, we evaluate the generative performance of
Gen\'eLive! using production datasets at KLab as well as open datasets for
reproducibility, while the model continues to operate in their business. Our
code and the model, tuned and trained using a supercomputer, are publicly
available.

该论文提出了一个基于深度生成模型的节奏动作游戏生成方法 Gen'eLive!，其应用可以减少业务操作成本，该模型通过考虑节拍和时间尺度提高了音乐结构的生成效果，并成功地应用于日本 KLab 公司《Love Live!》等游戏的制作中。

GenéLive！在 Love Live! 中生成节奏动作

GenéLive! Generating Rhythm Actions in Love Live!

We present a Temporal Context Network (TCN) for precise temporal localization
of human activities. Similar to the Faster-RCNN architecture, proposals are
placed at equal intervals in a video which span multiple temporal scales. We
propose a novel representation for ranking these proposals. Since pooling
features only inside a segment is not sufficient to predict activity
boundaries, we construct a representation which explicitly captures context
around a proposal for ranking it. For each temporal segment inside a proposal,
features are uniformly sampled at a pair of scales and are input to a temporal
convolutional neural network for classification. After ranking proposals,
non-maximum suppression is applied and classification is performed to obtain
final detections. TCN outperforms state-of-the-art methods on the ActivityNet
dataset and the THUMOS14 dataset.

该研究提出了一种基于 Temporal Context Network 的方法，能够精确地定位人类活动的时间，并在两个数据集上优于当今最先进的方法。