The project's aim is to create an AI agent capable of selecting good actions
in a game-playing domain called Battlespace. Sequential domains like
Battlespace are important testbeds for planning problems, as such, the
Department of Defense uses such domains for wargaming exercises. The agents we
developed combine Monte Carlo Tree Search (MCTS) and Deep Q-Network (DQN)
techniques in an effort to navigate the game environment, avoid obstacles,
interact with adversaries, and capture the flag. This paper will focus on the
encoding techniques we explored to present complex structured data stored in a
Python class, a necessary precursor to an agent.

该研究论文的主要内容是在一个名为 Battlespace 的游戏领域中创建一个能够选择良好行动的人工智能代理，使用 Monte Carlo Tree Search (MCTS) 和 Deep Q-Network (DQN) 技术，针对如何呈现存储在 Python 类中的复杂结构化数据进行了编码技术的探索。

对神经网络编码结构化数据的实验

Experiments with Encoding Structured Data for Neural Networks

In this study, we present two distinct approaches within the realm of Deep
Reinforcement Learning (Deep-RL) aimed at enhancing mapless navigation for a
ground-based mobile robot. The research methodology primarily involves a
comparative analysis between a Deep-RL strategy grounded in the foundational
Deep Q-Network (DQN) algorithm, and an alternative approach based on the Double
Deep Q-Network (DDQN) algorithm. The agents in these approaches leverage 24
measurements from laser range sampling, coupled with the agent's positional
differentials and orientation relative to the target. This amalgamation of data
influences the agents' determinations regarding navigation, ultimately
dictating the robot's velocities. By embracing this parsimonious sensory
framework as proposed, we successfully showcase the training of an agent for
proficiently executing navigation tasks and adeptly circumventing obstacles.
Notably, this accomplishment is attained without a dependency on intricate
sensory inputs like those inherent to image-centric methodologies. The proposed
methodology is evaluated in three different real environments, revealing that
Double Deep structures significantly enhance the navigation capabilities of
mobile robots compared to simple Q structures.

通过比较分析基于深度 Q 网络（DQN）算法和基于双重深度 Q 网络（DDQN）算法的深度强化学习（Deep-RL）策略，本研究提出了两种不同的方法，旨在增强地面移动机器人的无地图导航能力。这些方法中的智能体利用来自激光范围采样的 24 个测量，以及与目标的位置差异和方向相关的智能体位置数据，通过融合这些数据来影响导航决策，从而最终控制机器人的速度。实验结果表明，相比于简单的 Q 结构，双重深度结构显著提高了移动机器人的导航能力，且无需依赖于复杂的感知输入，如基于图像的方法。