Large Language Models exhibit robust problem-solving capabilities for diverse
tasks. However, most LLM-based agents are designed as specific task solvers
with sophisticated prompt engineering, rather than agents capable of learning
and evolving through interactions. These task solvers necessitate manually
crafted prompts to inform task rules and regulate LLM behaviors, inherently
incapacitating to address complex dynamic scenarios e.g., large interactive
games. In light of this, we propose Agent-Pro: an LLM-based Agent with
Policy-level Reflection and Optimization that can learn a wealth of expertise
from interactive experiences and progressively elevate its behavioral policy.
Specifically, it involves a dynamic belief generation and reflection process
for policy evolution. Rather than action-level reflection, Agent-Pro
iteratively reflects on past trajectories and beliefs, fine-tuning its
irrational beliefs for a better policy. Moreover, a depth-first search is
employed for policy optimization, ensuring continual enhancement in policy
payoffs. Agent-Pro is evaluated across two games: Blackjack and Texas Hold'em,
outperforming vanilla LLM and specialized models. Our results show Agent-Pro
can learn and evolve in complex and dynamic scenes, which also benefits
numerous LLM-based applications.

Agent-Pro 是基于 LLM 的智能代理，具备策略级别的反思和优化能力，可以通过互动经验不断学习和提升行为策略，在复杂和动态场景中表现出色。