The dispatch optimization of coal mine integrated energy system is
challenging due to high dimensionality, strong coupling constraints, and
multiobjective. Existing constrained multiobjective evolutionary algorithms
struggle with locating multiple small and irregular feasible regions, making
them inaplicable to this problem. To address this issue, we here develop a
multitask evolutionary algorithm framework that incorporates the dispatch
correlated domain knowledge to effectively deal with strong constraints and
multiobjective optimization. Possible evolutionary multitask construction
strategy based on complex constraint relationship analysis and handling, i.e.,
constraint coupled spatial decomposition, constraint strength classification
and constraint handling technique, is first explored. Within the multitask
evolutionary optimization framework, two strategies, i.e., an elite guided
knowledge transfer by designing a special crowding distance mechanism to select
dominant individuals from each task, and an adaptive neighborhood technology
based mutation to effectively balance the diversity and convergence of each
optimized task for the differential evolution algorithm, are further developed.
The performance of the proposed algorithm in feasibility, convergence, and
diversity is demonstrated in a case study of a coal mine integrated energy
system by comparing with CPLEX solver and seven constrained multiobjective
evolutionary algorithms.

煤矿综合能源系统的调度优化是一个具有高维度、强耦合约束和多目标的挑战性问题。针对此问题，我们开发了一个多任务进化算法框架，利用调度相关领域知识来有效处理强约束和多目标优化。通过复杂约束关系分析和处理，我们首先探索了一种可能的进化多任务构建策略，即约束耦合空间分解、约束强度分类和约束处理技术。在多任务进化优化框架中，进一步开发了两种策略，即通过设计特殊的拥挤距离机制从每个任务中选择主导个体的精英引导知识传递以及基于自适应邻域技术的突变，以有效平衡每个优化任务的多样性和收敛性。通过与 CPLEX 求解器和七种约束多目标进化算法的比较，在一个煤矿综合能源系统的案例研究中展示了所提出算法在可行性、收敛性和多样性方面的性能。

多任务多约束差分进化算法中基于优秀导向的知识传递在煤矿综合能源系统调度中的应用

Multi-task multi-constraint differential evolution with elite-guided  knowledge transfer for coal mine integrated energy system dispatching

We propose a novel hybrid stochastic policy gradient estimator by combining
an unbiased policy gradient estimator, the REINFORCE estimator, with another
biased one, an adapted SARAH estimator for policy optimization. The hybrid
policy gradient estimator is shown to be biased, but has variance reduced
property. Using this estimator, we develop a new Proximal Hybrid Stochastic
Policy Gradient Algorithm (ProxHSPGA) to solve a composite policy optimization
problem that allows us to handle constraints or regularizers on the policy
parameters. We first propose a single-looped algorithm then introduce a more
practical restarting variant. We prove that both algorithms can achieve the
best-known trajectory complexity $\mathcal{O}\left(\varepsilon^{-3}\right)$ to
attain a first-order stationary point for the composite problem which is better
than existing REINFORCE/GPOMDP $\mathcal{O}\left(\varepsilon^{-4}\right)$ and
SVRPG $\mathcal{O}\left(\varepsilon^{-10/3}\right)$ in the non-composite
setting. We evaluate the performance of our algorithm on several well-known
examples in reinforcement learning. Numerical results show that our algorithm
outperforms two existing methods on these examples. Moreover, the composite
settings indeed have some advantages compared to the non-composite ones on
certain problems.

提出了一种新的混合随机策略梯度估计器，并使用此估计器开发了一种新的混合随机策略梯度算法（ProxHSPGA）来解决复合策略优化问题，该算法可以处理策略参数上的约束或规则化，已经在强化学习中的一些示例上进行了评估和验证。

一种混合随机策略梯度算法用于强化学习

A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning

This paper considers the problem of minimizing an expectation function over a
closed convex set, coupled with a {\color{black} functional or expectation}
constraint on either decision variables or problem parameters. We first present
a new stochastic approximation (SA) type algorithm, namely the cooperative SA
(CSA), to handle problems with the constraint on devision variables. We show
that this algorithm exhibits the optimal ${\cal O}(1/\epsilon^2)$ rate of
convergence, in terms of both optimality gap and constraint violation, when the
objective and constraint functions are generally convex, where $\epsilon$
denotes the optimality gap and infeasibility. Moreover, we show that this rate
of convergence can be improved to ${\cal O}(1/\epsilon)$ if the objective and
constraint functions are strongly convex. We then present a variant of CSA,
namely the cooperative stochastic parameter approximation (CSPA) algorithm, to
deal with the situation when the constraint is defined over problem parameters
and show that it exhibits similar optimal rate of convergence to CSA. It is
worth noting that CSA and CSPA are primal methods which do not require the
iterations on the dual space and/or the estimation on the size of the dual
variables. To the best of our knowledge, this is the first time that such
optimal SA methods for solving functional or expectation constrained stochastic
optimization are presented in the literature.

本文介绍了两种新型随机逼近算法 —— 协作随机逼近算法（CSA）和协作随机参数逼近算法（CSPA），分别用于处理决策变量约束和问题参数约束的期望函数最小化问题，证明了它们实现了最优收敛率以及不需要使用对偶变量迭代。