With the increasing penetration of renewable power sources such as wind and
solar, accurate short-term, nowcasting renewable power prediction is becoming
increasingly important. This paper investigates the multi-modal (MM) learning
and end-to-end (E2E) learning for nowcasting renewable power as an intermediate
to energy management systems. MM combines features from all-sky imagery and
meteorological sensor data as two modalities to predict renewable power
generation that otherwise could not be combined effectively. The combined,
predicted values are then input to a differentiable optimal power flow (OPF)
formulation simulating the energy management. For the first time, MM is
combined with E2E training of the model that minimises the expected total
system cost. The case study tests the proposed methodology on the real sky and
meteorological data from the Netherlands. In our study, the proposed MM-E2E
model reduced system cost by 30% compared to uni-modal baselines.

本文探讨了多模态 (MM) 学习和端到端 (E2E) 学习方法，结合全天空影像和气象传感器数据以预测可再生能源发电量，并通过可微分的最优功率流 (OPF) 模拟能源管理，同时模型通过 E2E 训练以最小化预期总系统成本。在荷兰实际数据上的案例研究中发现，所提出的 MM-E2E 模型可以将系统成本降低 30％。