Accurate solar power forecasting is crucial to integrate photovoltaic plants
into the electric grid, schedule and secure the power grid safety. This problem
becomes more demanding for those newly installed solar plants which lack
sufficient data. Current research predominantly relies on historical solar
power data or numerical weather prediction in a single-modality format,
ignoring the complementary information provided in different modalities. In
this paper, we propose a multi-modality fusion framework to integrate
historical power data, numerical weather prediction, and satellite images,
significantly improving forecast performance. We introduce a vector quantized
framework that aligns modalities with varying information densities, striking a
balance between integrating sufficient information and averting model
overfitting. Our framework demonstrates strong zero-shot forecasting
capability, which is especially useful for those newly installed plants.
Moreover, we collect and release a multi-modal solar power (MMSP) dataset from
real-world plants to further promote the research of multi-modal solar
forecasting algorithms. Our extensive experiments show that our model not only
operates with robustness but also boosts accuracy in both zero-shot forecasting
and scenarios rich with training data, surpassing leading models. We have
incorporated it into our eForecaster platform and deployed it for more than 300
solar plants with a capacity of over 15GW.

利用历史功率数据、数值天气预报和卫星图像，提出了一个多模态融合框架，显著提高了太阳能功率预测性能，特别适用于新安装的太阳能发电厂。进行了广泛的实验表明，模型在零样本预测和训练数据丰富的场景下不仅具有稳健性，而且提高了准确性，超过了领先模型。已将其整合到 eForecaster 平台，用于超过 300 个装机容量超过 15GW 的太阳能电厂。