Most existing literature on supply chain and inventory management consider
stochastic demand processes with zero or constant lead times. While it is true
that in certain niche scenarios, uncertainty in lead times can be ignored, most
real-world scenarios exhibit stochasticity in lead times. These random
fluctuations can be caused due to uncertainty in arrival of raw materials at
the manufacturer's end, delay in transportation, an unforeseen surge in
demands, and switching to a different vendor, to name a few. Stochasticity in
lead times is known to severely degrade the performance in an inventory
management system, and it is only fair to abridge this gap in supply chain
system through a principled approach. Motivated by the recently introduced
delay-resolved deep Q-learning (DRDQN) algorithm, this paper develops a
reinforcement learning based paradigm for handling uncertainty in lead times
(\emph{action delay}). Through empirical evaluations, it is further shown that
the inventory management with uncertain lead times is not only equivalent to
that of delay in information sharing across multiple echelons
(\emph{observation delay}), a model trained to handle one kind of delay is
capable to handle delays of another kind without requiring to be retrained.
Finally, we apply the delay-resolved framework to scenarios comprising of
multiple products subjected to stochasticity in lead times, and elucidate how
the delay-resolved framework negates the effect of any delay to achieve
near-optimal performance.

本文考虑供应链中存在的交货期不确定性和因此导致的库存管理性能下降问题，提出了一种基于强化学习的延迟解决方法，并在多种不确定性的情景下进行了实证评估，证明该方法可以实现接近最优的性能。

多产品库存管理中处理不确定交货期的基于学习的框架

A Learning Based Framework for Handling Uncertain Lead Times in Multi-Product Inventory Management

We address the problem of production planning and distribution in
multi-echelon supply chains. We consider uncertain demands and lead times which
makes the problem stochastic and non-linear. A Markov Decision Process
formulation and a Non-linear Programming model are presented. As a sequential
decision-making problem, Deep Reinforcement Learning (RL) is a possible
solution approach. This type of technique has gained a lot of attention from
Artificial Intelligence and Optimization communities in recent years.
Considering the good results obtained with Deep RL approaches in different
areas there is a growing interest in applying them in problems from the
Operations Research field. We have used a Deep RL technique, namely Proximal
Policy Optimization (PPO2), to solve the problem considering uncertain, regular
and seasonal demands and constant or stochastic lead times. Experiments are
carried out in different scenarios to better assess the suitability of the
algorithm. An agent based on a linearized model is used as a baseline.
Experimental results indicate that PPO2 is a competitive and adequate tool for
this type of problem. PPO2 agent is better than baseline in all scenarios with
stochastic lead times (7.3-11.2%), regardless of whether demands are seasonal
or not. In scenarios with constant lead times, the PPO2 agent is better when
uncertain demands are non-seasonal (2.2-4.7%). The results show that the
greater the uncertainty of the scenario, the greater the viability of this type
of approach.

探讨了多级供应链中的生产计划和分配问题，利用深度增强学习技术 Proximal Policy Optimization（PPO2）来解决非线性不确定需求的问题，结果表明在有不确定性情况下，该方法更具优势。