Deep reinforcement learning (RL) has been shown to be effective in producing
approximate solutions to some vehicle routing problems (VRPs), especially when
using policies generated by encoder-decoder attention mechanisms. While these
techniques have been quite successful for relatively simple problem instances,
there are still under-researched and highly complex VRP variants for which no
effective RL method has been demonstrated. In this work we focus on one such
VRP variant, which contains multiple trucks and multi-leg routing requirements.
In these problems, demand is required to move along sequences of nodes, instead
of just from a start node to an end node. With the goal of making deep RL a
viable strategy for real-world industrial-scale supply chain logistics, we
develop new extensions to existing encoder-decoder attention models which allow
them to handle multiple trucks and multi-leg routing requirements. Our models
have the advantage that they can be trained for a small number of trucks and
nodes, and then embedded into a large supply chain to yield solutions for
larger numbers of trucks and nodes. We test our approach on a real supply chain
environment arising in the operations of Japanese automotive parts manufacturer
Aisin Corporation, and find that our algorithm outperforms Aisin's previous
best solution.

基于深度强化学习，使用编码器 - 解码器的注意机制生成的策略在某些车辆路径问题上表现出有效性，但在某些复杂车辆路径问题上，仍缺乏有效的强化学习方法。本研究针对一种包含多辆卡车和多段路径要求的车辆路径问题变体，扩展现有的编码器 - 解码器注意模型，使其能够处理多辆卡车和多段路径需求，并在日本汽车零部件制造商爱信公司的实际供应链环境中测试，发现我们的算法优于爱信公司的最佳解决方案。

多卡车多腿需求路线的深度强化学习

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems  with Multi-Leg Demand Routes

Recent work has shown that the encoder-decoder attention mechanisms in neural
machine translation (NMT) are different from the word alignment in statistical
machine translation. In this paper, we focus on analyzing encoder-decoder
attention mechanisms, in the case of word sense disambiguation (WSD) in NMT
models. We hypothesize that attention mechanisms pay more attention to context
tokens when translating ambiguous words. We explore the attention distribution
patterns when translating ambiguous nouns. Counter-intuitively, we find that
attention mechanisms are likely to distribute more attention to the ambiguous
noun itself rather than context tokens, in comparison to other nouns. We
conclude that attention mechanism is not the main mechanism used by NMT models
to incorporate contextual information for WSD. The experimental results suggest
that NMT models learn to encode contextual information necessary for WSD in the
encoder hidden states. For the attention mechanism in Transformer models, we
reveal that the first few layers gradually learn to "align" source and target
tokens and the last few layers learn to extract features from the related but
unaligned context tokens.

本文研究了神经机器翻译中编码器 - 解码器注意机制的特点，发现当翻译含义模糊的词汇时，注意机制更关注上下文标记。但与其他名词相比，注意机制更容易将更多注意力分配给模糊的名词本身而不是上下文标记，因此得出结论：注意机制不是神经机器翻译模型用于词义消歧所使用的主要机制之一。实验结果表明，NMT 模型在编码器隐藏状态中学习编码所需的上下文信息，而用于 Transformer 模型的注意机制表现为前几层逐渐了解源和目标标记，而最后几层学习从相关但不对齐的上下文标记中提取特征信息。