The learning and deployment of long-LLMs remains a challenging problem despite recent progresses. In this work, we argue that the long-LLMs are not a necessity to solve long-context tasks, as common long-context tasks are short-context solvable, i.e. they can be solved by purely working with oracle short-contexts within the long-context tasks' inputs. On top of this argument, we propose a framework called LC-Boost (Long-Context Bootstrapper), which enables a short-LLM to address the long-context tasks in a bootstrapping manner. In our framework, the short-LLM prompts itself to reason for two critical decisions: 1) how to access to the appropriate part of context within the input, 2) how to make effective use of the accessed context. By adaptively accessing and utilizing the context based on the presented tasks, LC-Boost can serve as a general framework to handle diversified long-context processing problems. We comprehensively evaluate different types of tasks from popular long-context benchmarks, where LC-Boost is able to achieve a substantially improved performance with a much smaller consumption of resource.

在这项工作中，我们认为虽然最近取得了一些进展，但长上下文语言模型（long-LLMs）的学习和部署仍然是一个具有挑战性的问题。我们提出了一个名为LC-Boost（长上下文引导器）的框架，该框架使得短上下文语言模型（short-LLM）能够以自主引导的方式解决长上下文任务。通过根据给出的任务自适应地访问和利用上下文，LC-Boost可以作为处理不同类型的长上下文处理问题的通用框架。我们从流行的长上下文基准中全面评估了不同类型的任务，在较小的资源消耗下，LC-Boost能够实现显著提高的性能。

长上下文任务是否需要长LLMs？