Orca 1 learns from rich signals, such as explanation traces, allowing it to
outperform conventional instruction-tuned models on benchmarks like BigBench
Hard and AGIEval. In Orca 2, we continue exploring how improved training
signals can enhance smaller LMs' reasoning abilities. Research on training
small LMs has often relied on imitation learning to replicate the output of
more capable models. We contend that excessive emphasis on imitation may
restrict the potential of smaller models. We seek to teach small LMs to employ
different solution strategies for different tasks, potentially different from
the one used by the larger model. For example, while larger models might
provide a direct answer to a complex task, smaller models may not have the same
capacity. In Orca 2, we teach the model various reasoning techniques
(step-by-step, recall then generate, recall-reason-generate, direct answer,
etc.). More crucially, we aim to help the model learn to determine the most
effective solution strategy for each task. We evaluate Orca 2 using a
comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
similar size and attains performance levels similar or better to those of
models 5-10x larger, as assessed on complex tasks that test advanced reasoning
abilities in zero-shot settings. We open-source Orca 2 to encourage further
research on the development, evaluation, and alignment of smaller LMs.

Orca 2 通过学习丰富的信号，诸如解释追踪等，使其在 BigBench Hard 和 AGIEval 等基准测试中优于传统的经过指导调整的模型。

Orca 2: 教授小型语言模型进行推理

Orca 2: Teaching Small Language Models How to Reason

In recent years, Bi-Level Optimization (BLO) techniques have received
extensive attentions from both learning and vision communities. A variety of
BLO models in complex and practical tasks are of non-convex follower structure
in nature (a.k.a., without Lower-Level Convexity, LLC for short). However, this
challenging class of BLOs is lack of developments on both efficient solution
strategies and solid theoretical guarantees. In this work, we propose a new
algorithmic framework, named Initialization Auxiliary and Pessimistic
Trajectory Truncated Gradient Method (IAPTT-GM), to partially address the above
issues. In particular, by introducing an auxiliary as initialization to guide
the optimization dynamics and designing a pessimistic trajectory truncation
operation, we construct a reliable approximate version of the original BLO in
the absence of LLC hypothesis. Our theoretical investigations establish the
convergence of solutions returned by IAPTT-GM towards those of the original BLO
without LLC. As an additional bonus, we also theoretically justify the quality
of our IAPTT-GM embedded with Nesterov's accelerated dynamics under LLC. The
experimental results confirm both the convergence of our algorithm without LLC,
and the theoretical findings under LLC.

本文提出了一种名为 IAPTT-GM 的算法框架，旨在解决具有非凸跟随结构的 Bi-Level Optimization 模型在解决高复杂度实际问题时的有效解决方案和可靠的近似版本建构问题，并通过理论研究证明了该算法的收敛性和运用 LLC 时 Nesterov 加速动力学的质量。

基于梯度的非凸跟随者双层优化及其拓展

Towards Gradient-based Bilevel Optimization with Non-convex Followers  and Beyond

Bilevel optimization is defined as a mathematical program, where an
optimization problem contains another optimization problem as a constraint.
These problems have received significant attention from the mathematical
programming community. Only limited work exists on bilevel problems using
evolutionary computation techniques; however, recently there has been an
increasing interest due to the proliferation of practical applications and the
potential of evolutionary algorithms in tackling these problems. This paper
provides a comprehensive review on bilevel optimization from the basic
principles to solution strategies; both classical and evolutionary. A number of
potential application problems are also discussed. To offer the readers
insights on the prominent developments in the field of bilevel optimization, we
have performed an automated text-analysis of an extended list of papers
published on bilevel optimization to date. This paper should motivate
evolutionary computation researchers to pay more attention to this practical
yet challenging area.

本文介绍了双层优化的基本原理和解决策略，包括经典方法和进化计算技术，并讨论了一些可能的应用问题，鼓励进化计算研究人员更多关注这个实用而具有挑战性的领域。