The transformer is a neural network component that can be used to learn
useful representations of sequences or sets of datapoints. The transformer has
driven recent advances in natural language processing, computer vision, and
spatio-temporal modelling. There are many introductions to transformers, but
most do not contain precise mathematical descriptions of the architecture and
the intuitions behind the design choices are often also missing. Moreover, as
research takes a winding path, the explanations for the components of the
transformer can be idiosyncratic. In this note we aim for a mathematically
precise, intuitive, and clean description of the transformer architecture.

本文将准确而简洁介绍变形器体系结构的数学原理，以及设计决策背后的洞察。

Transformer 简介

An Introduction to Transformers

As the costs of sensors and associated IT infrastructure decreases - as
exemplified by the Internet of Things - increasing volumes of observational
data are becoming available for use by environmental scientists. However, as
the number of available observation sites increases, so too does the
opportunity for data quality issues to emerge, particularly given that many of
these sensors do not have the benefit of official maintenance teams. To realise
the value of crowd sourced 'Internet of Things' type observations for
environmental modelling, we require approaches that can automate the detection
of outliers during the data modelling process so that they do not contaminate
the true distribution of the phenomena of interest. To this end, here we
present a Bayesian deep learning approach for spatio-temporal modelling of
environmental variables with automatic outlier detection. Our approach
implements a Gaussian-uniform mixture density network whose dual purposes -
modelling the phenomenon of interest, and learning to classify and ignore
outliers - are achieved simultaneously, each by specifically designed branches
of our neural network. For our example application, we use the Met Office's
Weather Observation Website data, an archive of observations from around 1900
privately run and unofficial weather stations across the British Isles. Using
data on surface air temperature, we demonstrate how our deep mixture model
approach enables the modelling of a highly skilled spatio-temporal temperature
distribution without contamination from spurious observations. We hope that
adoption of our approach will help unlock the potential of incorporating a
wider range of observation sources, including from crowd sourcing, into future
environmental models.

通过引入 Bayesian 深度学习的高斯 - 均匀混合密度网络，实现对环境变量在时空域上的建模和异常值检测，为处理来自 ' 物联网 ' 观测点数据的环境建模提供了途径。