We introduce EarthPT -- an Earth Observation (EO) pretrained transformer.
EarthPT is a 700 million parameter decoding transformer foundation model
trained in an autoregressive self-supervised manner and developed specifically
with EO use-cases in mind. We demonstrate that EarthPT is an effective
forecaster that can accurately predict future pixel-level surface reflectances
across the 400-2300 nm range well into the future. For example, forecasts of
the evolution of the Normalised Difference Vegetation Index (NDVI) have a
typical error of approximately 0.05 (over a natural range of -1 -> 1) at the
pixel level over a five month test set horizon, out-performing simple
phase-folded models based on historical averaging. We also demonstrate that
embeddings learnt by EarthPT hold semantically meaningful information and could
be exploited for downstream tasks such as highly granular, dynamic land use
classification. Excitingly, we note that the abundance of EO data provides us
with -- in theory -- quadrillions of training tokens. Therefore, if we assume
that EarthPT follows neural scaling laws akin to those derived for Large
Language Models (LLMs), there is currently no data-imposed limit to scaling
EarthPT and other similar `Large Observation Models.'

EarthPT 是一个地球观测预训练的变压器模型，通过自回归自监督方式训练了一个 7 亿参数、针对地球观测应用场景的模型。我们展示了 EarthPT 是一个有效的预测器，可以准确预测未来的像素级表面反射率在 400-2300nm 范围内变化。此外，通过 EarthPT 学习的嵌入具有语义上有意义的信息，可用于下游任务，如高精度、动态的土地利用分类。