The modeling of environmental ecosystems plays a pivotal role in the
sustainable management of our planet. Accurate prediction of key environmental
variables over space and time can aid in informed policy and decision-making,
thus improving people's livelihood. Recently, deep learning-based methods have
shown promise in modeling the spatial-temporal relationships for predicting
environmental variables. However, these approaches often fall short in handling
incomplete features and distribution shifts, which are commonly observed in
environmental data due to the substantial cost of data collection and
malfunctions in measuring instruments. To address these issues, we propose LITE
-- a multimodal large language model for environmental ecosystems modeling.
Specifically, LITE unifies different environmental variables by transforming
them into natural language descriptions and line graph images. Then, LITE
utilizes unified encoders to capture spatial-temporal dynamics and correlations
in different modalities. During this step, the incomplete features are imputed
by a sparse Mixture-of-Experts framework, and the distribution shift is handled
by incorporating multi-granularity information from past observations. Finally,
guided by domain instructions, a language model is employed to fuse the
multimodal representations for the prediction. Our experiments demonstrate that
LITE significantly enhances performance in environmental spatial-temporal
prediction across different domains compared to the best baseline, with a
41.25% reduction in prediction error. This justifies its effectiveness. Our
data and code are available at this https URL

提出了一种用于环境生态系统建模的多模式大型语言模型 LITE，通过将不同的环境变量转化为自然语言描述和折线图像来统一这些变量，并利用统一编码器来捕捉不同模态的空间 - 时间动态和相关性，借助领域指令来融合多模态表示进行预测，从而显著提高了环境空间 - 时间预测的性能，相较于最佳基准线，预测误差减少了 41.25%。