Large language models (LLMs) have shown great promise at generating robot
programs from natural language given domain-specific robot application
programming interfaces (APIs). However, the performance gap between proprietary
LLMs and smaller open-weight LLMs remains wide. This raises a question: Can we
fine-tune smaller open-weight LLMs for generating domain-specific robot
programs to close the performance gap with proprietary LLMs? While
Self-Instruct is a promising solution by generating a diverse set of training
data, it cannot verify the correctness of these programs. In contrast, a robot
simulator with a well-defined world can identify execution errors but limits
the diversity of programs that it can verify. In this work, we introduce
Robo-Instruct, which brings the best of both worlds -- it promotes the
diversity of Self-Instruct while providing the correctness of simulator-based
checking. Robo-Instruct introduces RoboSim to synthesize a consistent world
state on the fly by inferring properties relevant to the program being checked,
and simulating actions accordingly. Furthermore, the instructions and programs
generated by Self-Instruct may be subtly inconsistent -- such as the program
missing a step implied by the instruction. Robo-Instruct further addresses this
with InstAlign, an instruction-program alignment procedure that revises the
task instruction to reflect the actual results of the generated program. Given
a few seed task descriptions and the robot APIs, Robo-Instruct is capable of
generating a training dataset using only a small open-weight model. This
dataset can then be used to fine-tune small open-weight language models,
enabling them to match or even exceed the performance of several proprietary
LLMs, such as GPT-3.5-Turbo and Gemini-Pro.

通过使用 Robo-Instruct 方法，我们可以使用小型的开源权重语言模型生成域特定的机器人程序，以实现与专有权重语言模型相媲美甚至超过其性能。

机器教学：增强了指令对齐的代码 LLM 微调的模拟器

Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning  CodeLLMs

This paper presents a new tool learning dataset Seal-Tools, which contains
self-instruct API-like tools. Seal-Tools not only offers a large number of
tools, but also includes instances which demonstrate the practical application
of tools. Seeking to generate data on a large scale while ensuring reliability,
we propose a self-instruct method to generate tools and instances, allowing
precise control over the process. Moreover, our Seal-Tools contains hard
instances that call multiple tools to complete the job, among which some are
nested tool callings. For precise and comprehensive evaluation, we use strict
format control and design three metrics from different dimensions. Therefore,
Seal-Tools can serve as a new benchmark to evaluate the tool-calling ability of
LLMs. Finally, we evaluate several prevalent LLMs and our finetuned model on
Seal-Tools. The results show that current systems are far from perfect. The
code, data and experiment results are available at
this https URL .

本研究提出了一个新的工具学习数据集 Seal-Tools，它包含了自学习的类似 API 的工具。通过提供大量的工具和实例，Seal-Tools 不仅能实现数据的大规模生成，还能展示工具的实际应用。为了确保可靠性，我们提出了一种自学习方法来生成工具和实例，使得过程能够被精确控制。此外，Seal-Tools 包含一些难度较高的实例，其中有些实例需要调用多个嵌套的工具来完成任务。为了进行精确和综合的评估，我们采用了严格的格式控制，并从不同角度设计了三个评估指标。因此，Seal-Tools 可以作为评估 LLMs 的工具调用能力的新基准。最后，我们在 Seal-Tools 上评估了几种流行的 LLMs 和我们微调的模型。结果表明当前的系统还远未达到完美。解码、数据和实验结果可在此链接获取。

Seal-Tools: 自助工具学习数据集用于代理调整和详细基准测试

Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and  Detailed Benchmark

Instruction tuning plays a pivotal role in Code Large Language Models (Code
LLMs) for the task of program synthesis. Presently, two dominant paradigms for
collecting tuning data are natural-instruct (human-written) and self-instruct
(automatically generated). Natural-instruct includes diverse and correct codes
but lacks instruction-code pairs, and exists improper code formats like nested
single-line codes. In contrast, self-instruct automatically generates proper
paired data. However, it suffers from low diversity due to generating
duplicates and cannot ensure the correctness of codes. To bridge the both
paradigms, we propose \textbf{Semi-Instruct}. It first converts diverse but
improper codes from natural-instruct into proper instruction-code pairs through
a method similar to self-instruct. To verify the correctness of generated
codes, we design a novel way to construct test cases by generating cases'
inputs and executing correct codes from natural-instruct to get outputs.
Finally, diverse and correct instruction-code pairs are retained for
instruction tuning. Experiments show that semi-instruct is significantly better
than natural-instruct and self-instruct. Furthermore, the performance steadily
improves as data scale increases.

指令调优在大规模代码语言模型（Code LLMs）中对程序合成任务至关重要。本文提出一种名为 Semi-Instruct 的方法，通过将自然指令中不规范代码转化为正确的指令 - 代码对，并设计了一种新颖的测试用例构建方法来验证生成代码的正确性。实验结果显示，Semi-Instruct 方法在数据规模增加时性能稳定提高。

半指导：连接自然指导和自我指导的大型编码语言模型

Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code  Large Language Models

Large "instruction-tuned" language models (finetuned to respond to
instructions) have demonstrated a remarkable ability to generalize zero-shot to
new tasks. Nevertheless, they depend heavily on human-written instruction data
that is limited in quantity, diversity, and creativity, therefore hindering the
generality of the tuned model. We introduce Self-Instruct, a framework for
improving the instruction-following capabilities of pretrained language models
by bootstrapping off its own generations. Our pipeline generates instruction,
input, and output samples from a language model, then prunes them before using
them to finetune the original model. Applying our method to vanilla GPT3, we
demonstrate a 33% absolute improvement over the original model on
Super-NaturalInstructions, on par with the performance of InstructGPT_001,
which is trained with private user data and human annotations. For further
evaluation, we curate a set of expert-written instructions for novel tasks, and
show through human evaluation that tuning GPT3 with Self-Instruct outperforms
using existing public instruction datasets by a large margin, leaving only a 5%
absolute gap behind InstructGPT_001. Self-Instruct provides an almost
annotation-free method for aligning pre-trained language models with
instructions, and we release our large synthetic dataset to facilitate future
studies on instruction tuning.

使用 Self-Instruct 框架，可以将预训练模型与指令对齐，提高模型的指令遵循能力，无需过多人为指令数据，可用于 finetuning，大大提高了模型的通用性。