Prediction and optimisation are two widely used techniques that have found
many applications in solving real-world problems. While prediction is concerned
with estimating the unknown future values of a variable, optimisation is
concerned with optimising the decision given all the available data. These
methods are used together to solve problems for sequential decision-making
where often we need to predict the future values of variables and then use them
for determining the optimal decisions. This paradigm is known as forecast and
optimise and has numerous applications, e.g., forecast demand for a product and
then optimise inventory, forecast energy demand and schedule generations,
forecast demand for a service and schedule staff, to name a few. In this
extended abstract, we review a digital twin that was developed and applied in
wastewater treatment in Urban Utility to improve their operational efficiency.
While the current study is tailored to the case study problem, the underlying
principles can be used to solve similar problems in other domains.

利用预测和优化技术可以提高城市公共事业中污水处理的操作效率。

利用机器学习的数字化孪生模型用于预测和决策优化：在污水处理中的应用

Digital Twins for forecasting and decision optimisation with machine  learning: applications in wastewater treatment

Even though Deep Reinforcement Learning (DRL) showed outstanding results in
the fields of Robotics and Games, it is still challenging to implement it in
the optimization of industrial processes like wastewater treatment. One of the
challenges is the lack of a simulation environment that will represent the
actual plant as accurately as possible to train DRL policies. Stochasticity and
non-linearity of wastewater treatment data lead to unstable and incorrect
predictions of models over long time horizons. One possible reason for the
models' incorrect simulation behavior can be related to the issue of
compounding error, which is the accumulation of errors throughout the
simulation. The compounding error occurs because the model utilizes its
predictions as inputs at each time step. The error between the actual data and
the prediction accumulates as the simulation continues. We implemented two
methods to improve the trained models for wastewater treatment data, which
resulted in more accurate simulators: 1- Using the model's prediction data as
input in the training step as a tool of correction, and 2- Change in the loss
function to consider the long-term predicted shape (dynamics). The experimental
results showed that implementing these methods can improve the behavior of
simulators in terms of Dynamic Time Warping throughout a year up to 98%
compared to the base model. These improvements demonstrate significant promise
in creating simulators for biological processes that do not need pre-existing
knowledge of the process but instead depend exclusively on time series data
obtained from the system.

在废水处理中，尽管深度强化学习在机器人技术和游戏领域取得了杰出的成果，但在工业流程优化方面仍面临挑战。本研究通过使用模型的预测数据作为输入以进行校正，并改变损失函数以考虑长期预测形态（动态），提供了改进废水处理数据训练模型的两种方法，提高了模拟器在一年中动态时间弯曲方面的性能。这些改进为基于时间序列数据的生物过程模拟器的创建提供了重要的希望。

改进的基于长短期记忆模型的深度强化学习废水处理模拟器

Improved Long Short-Term Memory-based Wastewater Treatment Simulators  for Deep Reinforcement Learning

Phosphorus removal is vital in wastewater treatment to reduce reliance on
limited resources. Deep reinforcement learning (DRL) is a machine learning
technique that can optimize complex and nonlinear systems, including the
processes in wastewater treatment plants, by learning control policies through
trial and error. However, applying DRL to chemical and biological processes is
challenging due to the need for accurate simulators. This study trained six
models to identify the phosphorus removal process and used them to create a
simulator for the DRL environment. Although the models achieved high accuracy
(>97%), uncertainty and incorrect prediction behavior limited their performance
as simulators over longer horizons. Compounding errors in the models'
predictions were identified as one of the causes of this problem. This approach
for improving process control involves creating simulation environments for DRL
algorithms, using data from supervisory control and data acquisition (SCADA)
systems with a sufficient historical horizon without complex system modeling or
parameter estimation.

通过训练六个模型，创建了深度强化学习 (DRL) 的仿真环境，利用监控与数据采集系统 (SCADA) 提供的历史数据来改进废水处理过程中的磷去除，并解决了由模型预测误差累积引起的问题。