In this paper, we introduce Kun, a novel approach for creating high-quality
instruction-tuning datasets for large language models (LLMs) without relying on
manual annotations. Adapting a self-training algorithm based on instruction
back-translation and answer polishment, Kun leverages unlabelled data from
diverse sources such as Wudao, Wanjuan, and SkyPile to generate a substantial
dataset of over a million Chinese instructional data points. This approach
significantly deviates from traditional methods by using a self-curation
process to refine and select the most effective instruction-output pairs. Our
experiments with the 6B-parameter Yi model across various benchmarks
demonstrate Kun's robustness and scalability. Our method's core contributions
lie in its algorithmic advancement, which enhances data retention and clarity,
and its innovative data generation approach that substantially reduces the
reliance on costly and time-consuming manual annotations. This methodology
presents a scalable and efficient solution for improving the
instruction-following capabilities of LLMs, with significant implications for
their application across diverse fields. The code and dataset can be found at
this https URL

采用自我训练算法通过自给自足的方式生成高质量的指令调整数据集，为大规模语言模型的指令追踪能力提供一种可扩展和高效的解决方案。

Kun: 中文自对齐指导下的答案优化

Kun: Answer Polishment for Chinese Self-Alignment with Instruction  Back-Translation

Weakly supervised learning has emerged as a compelling tool for object
detection by reducing the need for strong supervision during training. However,
major challenges remain: (1) differentiation of object instances can be
ambiguous; (2) detectors tend to focus on discriminative parts rather than
entire objects; (3) without ground truth, object proposals have to be redundant
for high recalls, causing significant memory consumption. Addressing these
challenges is difficult, as it often requires to eliminate uncertainties and
trivial solutions. To target these issues we develop an instance-aware and
context-focused unified framework. It employs an instance-aware self-training
algorithm and a learnable Concrete DropBlock while devising a memory-efficient
sequential batch back-propagation. Our proposed method achieves
state-of-the-art results on COCO ($12.1\% ~AP$, $24.8\% ~AP_{50}$), VOC 2007
($54.9\% ~AP$), and VOC 2012 ($52.1\% ~AP$), improving baselines by great
margins. In addition, the proposed method is the first to benchmark ResNet
based models and weakly supervised video object detection. Code, models, and
more details will be made available at: this https URL

该研究提出了一种实例感知和以上下文为中心的统一框架，采用实例感知自训练算法和可学习 Concrete DropBlock，同时设计了一种内存高效的顺序批次反向传播。该方法在 COCO、VOC2007 和 VOC2012 数据集上取得了最先进的结果，并且是第一个基于 ResNet 的研究，还进行了弱监督视频目标检测的基准测试。

基于实例感知、上下文关注和内存高效的弱监督目标检测

Instance-aware, Context-focused, and Memory-efficient Weakly Supervised  Object Detection

Neural dependency parsing has proven very effective, achieving
state-of-the-art results on numerous domains and languages. Unfortunately, it
requires large amounts of labeled data, that is costly and laborious to create.
In this paper we propose a self-training algorithm that alleviates this
annotation bottleneck by training a parser on its own output. Our Deep
Contextualized Self-training (DCST) algorithm utilizes representation models
trained on sequence labeling tasks that are derived from the parser's output
when applied to unlabeled data, and integrates these models with the base
parser through a gating mechanism. We conduct experiments across multiple
languages, both in low resource in-domain and in cross-domain setups, and
demonstrate that DCST substantially outperforms traditional self-training as
well as recent semi-supervised training methods.

本文提出了一种自我训练算法 Deep Contextualized Self-training (DCST)，利用序列标注的表示模型来解决大量标注数据的问题，并通过控制机制将这些模型集成到基础解析器中，并在多语言实验中取得了优异的成绩。