Efficiently utilizing rich knowledge in pretrained models has become a
critical topic in the era of large models. This work focuses on adaptively
utilizing knowledge from multiple source-pretrained models to an unlabeled
target domain without accessing the source data. Despite being a practically
useful setting, existing methods require extensive parameter tuning over each
source model, which is computationally expensive when facing abundant source
domains or larger source models. To address this challenge, we propose a novel
approach which is free of the parameter tuning over source backbones. Our
technical contribution lies in the Bi-level ATtention ENsemble (Bi-ATEN)
module, which learns both intra-domain weights and inter-domain ensemble
weights to achieve a fine balance between instance specificity and domain
consistency. By slightly tuning source bottlenecks, we achieve comparable or
even superior performance on a challenging benchmark DomainNet with less than
3% trained parameters and 8 times of throughput compared with SOTA method.
Furthermore, with minor modifications, the proposed module can be easily
equipped to existing methods and gain more than 4% performance boost. Code is
available at this https URL

通过使用多个预训练模型中的丰富知识来适应地利用知识是大模型时代中的一个重要问题。本研究提出了一种新方法，通过 Bi-ATEN 模块来学习领域内权重和领域间组合权重，以在实例的特殊性和域的一致性之间实现良好平衡，从而实现在无需访问源数据的未标记目标域中自适应地利用来自多个源预训练模型的知识。

敏捷多源无域适应

Agile Multi-Source-Free Domain Adaptation

Large language models (LLMs) are primarily evaluated by overall performance
on various text understanding and generation tasks. However, such a paradigm
fails to comprehensively differentiate the fine-grained language and cognitive
skills, rendering the lack of sufficient interpretation to LLMs' capabilities.
In this paper, we present FAC$^2$E, a framework for Fine-grAined and
Cognition-grounded LLMs' Capability Evaluation. Specifically, we formulate
LLMs' evaluation in a multi-dimensional and explainable manner by dissociating
the language-related capabilities and the cognition-related ones. Besides,
through extracting the intermediate reasoning from LLMs, we further break down
the process of applying a specific capability into three sub-steps: recalling
relevant knowledge, utilizing knowledge, and solving problems. Finally,
FAC$^2$E evaluates each sub-step of each fine-grained capability, providing a
two-faceted diagnosis for LLMs. Utilizing FAC$^2$E, we identify a common
shortfall in knowledge utilization among models and propose a straightforward,
knowledge-enhanced method to mitigate this issue. Our results not only showcase
promising performance enhancements but also highlight a direction for future
LLM advancements.

FAC$^2$E 是一个针对大型语言模型 (LLMs) 的能力评估框架，通过提取 LLMs 的中间推理，将特定能力应用过程分解为三个子步骤，并评估每个细分能力的各个子步骤，从而全面区分 LLMs 的语言相关能力和认知相关能力。利用 FAC$^2$E，我们发现模型中知识利用方面存在常见不足，并提出了一种简单、知识增强的方法来缓解这个问题。我们的研究不仅展示了有希望的性能改进，还为未来 LLMs 的发展方向提供了启示。

FAC$^2$E：通过区分语言和认知来更好地理解大型语言模型的能力

FAC$^2$E: Better Understanding Large Language Model Capabilities by  Dissociating Language and Cognition

Retrieval-augmented language models (RALMs) improve performance by accessing
long-tail and up-to-date knowledge from external data stores, but are
challenging to build. Existing approaches require either expensive
retrieval-specific modifications to LM pre-training or use post-hoc integration
of the data store that leads to suboptimal performance. We introduce
Retrieval-Augmented Dual Instruction Tuning (RA-DIT), a lightweight fine-tuning
methodology that provides a third option by retrofitting any LLM with retrieval
capabilities. Our approach operates in two distinct fine-tuning steps: (1) one
updates a pre-trained LM to better use retrieved information, while (2) the
other updates the retriever to return more relevant results, as preferred by
the LM. By fine-tuning over tasks that require both knowledge utilization and
contextual awareness, we demonstrate that each stage yields significant
performance improvements, and using both leads to additional gains. Our best
model, RA-DIT 65B, achieves state-of-the-art performance across a range of
knowledge-intensive zero- and few-shot learning benchmarks, significantly
outperforming existing in-context RALM approaches by up to +8.9% in 0-shot
setting and +1.4% in 5-shot setting on average.

Retrieval-augmented language models can be enhanced through a lightweight fine-tuning methodology called RA-DIT, which effectively integrates retrieval capabilities into any pre-trained language model, resulting in state-of-the-art performance in zero- and few-shot learning scenarios.