Large language models (LLMs) often struggle to provide up-to-date information
due to their one-time training and the constantly evolving nature of the world.
To keep LLMs current, existing approaches typically involve continued
pre-training on new documents. However, they frequently face difficulties in
extracting stored knowledge. Motivated by the remarkable success of the Feynman
Technique in efficient human learning, we introduce Self-Tuning, a learning
framework aimed at improving an LLM's ability to effectively acquire new
knowledge from raw documents through self-teaching. Specifically, we develop a
Self-Teaching strategy that augments the documents with a set of
knowledge-intensive tasks created in a self-supervised manner, focusing on
three crucial aspects: memorization, comprehension, and self-reflection.
Additionally, we introduce three Wiki-Newpages-2023-QA datasets to facilitate
an in-depth analysis of an LLM's knowledge acquisition ability concerning
memorization, extraction, and reasoning. Extensive experimental results on
Llama2 family models reveal that Self-Tuning consistently exhibits superior
performance across all knowledge acquisition tasks and excels in preserving
previous knowledge.

通过自学习的方式，引入 Self-Tuning 学习框架以提高大型语言模型的新知识获取能力，通过加强文档的自监督任务，着重在记忆、理解和自反思三个关键方面，实现从原始文档中高效获取新知识，通过三个数据集进行对知识获取能力的深入分析，实验证明 Self-Tuning 在所有知识获取任务上保持出色的性能，并擅长保留先前的知识。

自动调整：通过自学指导 LLMs 有效获得新知识

Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge  through Self-Teaching

Deep learning has made revolutionary advances to diverse applications in the
presence of large-scale labeled datasets. However, it is prohibitively
time-costly and labor-expensive to collect sufficient labeled data in most
realistic scenarios. To mitigate the requirement for labeled data,
semi-supervised learning (SSL) focuses on simultaneously exploring both labeled
and unlabeled data, while transfer learning (TL) popularizes a favorable
practice of fine-tuning a pre-trained model to the target data. A dilemma is
thus encountered: Without a decent pre-trained model to provide an implicit
regularization, SSL through self-training from scratch will be easily misled by
inaccurate pseudo-labels, especially in large-sized label space; Without
exploring the intrinsic structure of unlabeled data, TL through fine-tuning
from limited labeled data is at risk of under-transfer caused by model shift.
To escape from this dilemma, we present Self-Tuning to enable data-efficient
deep learning by unifying the exploration of labeled and unlabeled data and the
transfer of a pre-trained model, as well as a Pseudo Group Contrast (PGC)
mechanism to mitigate the reliance on pseudo-labels and boost the tolerance to
false labels. Self-Tuning outperforms its SSL and TL counterparts on five tasks
by sharp margins, e.g. it doubles the accuracy of fine-tuning on Cars with 15%
labels.

本文提出了 Self-Tuning 模型，通过自动调整来联合探索有标签和无标签数据以及预训练模型的转移，并且使用 Pseudo Group Contrast 机制来缓解对伪标签的依赖性和提高对错误标签的容忍度。实验表明，在五个任务中，Self-Tuning 具有比其 SSL 和 TL 对应物更好的表现，例如在 15% 标签的 Cars 数据集上将精度提升了一倍。

数据高效深度学习的自动调整方法

Self-Tuning for Data-Efficient Deep Learning

The long-Term Evolution (LTE) of the 3GPP (3rd Generation Partnership
Project) radio access network is in early stage of specification. Self-tuning
and self-optimisation algorithms are currently studied with the aim of
enriching the LTE standard. This paper investigates auto-tuning of LTE mobility
algorithm. The auto-tuning is carried out by adapting handover parameters of
each base station according to its radio load and the load of its adjacent
cells. The auto-tuning alleviates cell congestion and balances the traffic and
the load between cells by handing off mobiles close to the cell border from the
congested cell to its neighbouring cells. Simulation results show that the
auto-tuning process brings an important gain in both call admission rate and
user throughput.

本研究探讨了 LTE 移动性算法的自动调优，通过根据基站的无线负载和相邻单元的负载，调整基站的切换参数，实现了单元之间的流量与负载均衡。模拟结果表明，自动调优过程在呼叫接纳率和用户吞吐量方面都带来了重要的增益。