The rapid progress in Multimodal Large Language Models (MLLMs) has
significantly advanced their ability to process and understand complex visual
and textual information. However, the integration of multiple images and
extensive textual contexts remains a challenge due to the inherent limitation
of the models' capacity to handle long input sequences efficiently. In this
paper, we introduce SEEKER, a multimodal large language model designed to
tackle this issue. SEEKER aims to optimize the compact encoding of long text by
compressing the text sequence into the visual pixel space via images, enabling
the model to handle long text within a fixed token-length budget efficiently.
Our empirical experiments on six long-context multimodal tasks demonstrate that
SEEKER can leverage fewer image tokens to convey the same amount of textual
information compared with the OCR-based approach, and is more efficient in
understanding long-form multimodal input and generating long-form textual
output, outperforming all existing proprietary and open-source MLLMs by large
margins.

通过将文本序列压缩到视觉像素空间中的图像，SEEKER 旨在优化长文本的紧凑编码，以便于在固定的令牌长度预算内高效处理长文本，并在理解长格式多模输入和生成长格式文本输出方面胜过所有现有专有和开源 MLLMs。

从文本到像素：MLLMs 中的长篇背景理解的进展

From Text to Pixel: Advancing Long-Context Understanding in MLLMs

This paper introduces Seeker, a system that allows users to interactively
refine search rankings in real time, through feedback in the form of likes and
dislikes. When searching online, users may not know how to accurately describe
their product of choice in words. An alternative approach is to search an
embedding space, allowing the user to query using a representation of the item
(like a tune for a song, or a picture for an object). However, this approach
requires the user to possess an example representation of their desired item.
Additionally, most current search systems do not allow the user to dynamically
adapt the results with further feedback. On the other hand, users often have a
mental picture of the desired item and are able to answer ordinal questions of
the form: "Is this item similar to what you have in mind?" With this
assumption, our algorithm allows for users to provide sequential feedback on
search results to adapt the search feed. We show that our proposed approach
works well both qualitatively and quantitatively. Unlike most previous
representation-based search systems, we can quantify the quality of our
algorithm by evaluating humans-in-the-loop experiments.

该论文介绍了 Seeker 系统，它允许用户通过喜欢和不喜欢的反馈形式实时交互地细化搜索排名。用户可以通过一个对象的嵌入空间进行搜索，然后采用顺序反馈逐步适应搜索结果。该算法可以通过人类参与的实验量化评估，从而使得该方法具有质量保证。