The notable success of large language models (LLMs) has sparked an upsurge in
building language agents to complete various complex tasks. We present AMOR, an
agent framework based on open-source LLMs, which reasons with external
knowledge bases and adapts to specific domains through human supervision to the
reasoning process. AMOR builds reasoning logic over a finite state machine
(FSM) that solves problems through autonomous executions and transitions over
disentangled modules. This allows humans to provide direct feedback to the
individual modules, and thus naturally forms process supervision. Based on this
reasoning and feedback framework, we develop AMOR through two-stage
fine-tuning: warm-up and adaptation. The former fine-tunes the LLM with
examples automatically constructed from various public datasets and enables
AMOR to generalize across different knowledge environments, while the latter
tailors AMOR to specific domains using process feedback. Extensive experiments
across multiple domains demonstrate the advantage of AMOR to strong baselines,
thanks to its FSM-based reasoning and process feedback mechanism.

基于开源语言模型的 AMOR 代理框架通过外部知识库进行推理，并通过人类监督适应于具体领域的推理过程，利用有限状态机构建推理逻辑以实现问题解决和模块之间的过渡，并通过两阶段微调加强其泛化能力和适应性。通过多个领域的广泛实验，证明了基于有限状态机推理和过程反馈机制的 AMOR 相对于强基线的优势。