We consider sequential treatment regimes where each unit is exposed to
combinations of interventions over time. When interventions are described by
qualitative labels, such as ``close schools for a month due to a pandemic'' or
``promote this podcast to this user during this week'', it is unclear which
appropriate structural assumptions allow us to generalize behavioral
predictions to previously unseen combinatorial sequences. Standard black-box
approaches mapping sequences of categorical variables to outputs are
applicable, but they rely on poorly understood assumptions on how reliable
generalization can be obtained, and may underperform under sparse sequences,
temporal variability, and large action spaces. To approach that, we pose an
explicit model for \emph{composition}, that is, how the effect of sequential
interventions can be isolated into modules, clarifying which data conditions
allow for the identification of their combined effect at different units and
time steps. We show the identification properties of our compositional model,
inspired by advances in causal matrix factorization methods but focusing on
predictive models for novel compositions of interventions instead of matrix
completion tasks and causal effect estimation. We compare our approach to
flexible but generic black-box models to illustrate how structure aids
prediction in sparse data conditions.

我们提出了一个明确的模型来解构顺序干预的效果，并研究其在预测模型中的识别性质，与传统的泛化方法相比，结构化模型在稀疏数据条件下更有助于预测。

组合顺序干预的结构化学习

Structured Learning of Compositional Sequential Interventions

We consider the problem of sequential change detection, where the goal is to
design a scheme for detecting any changes in a parameter or functional $\theta$
of the data stream distribution that has small detection delay, but guarantees
control on the frequency of false alarms in the absence of changes. In this
paper, we describe a simple reduction from sequential change detection to
sequential estimation using confidence sequences: we begin a new
$(1-\alpha)$-confidence sequence at each time step, and proclaim a change when
the intersection of all active confidence sequences becomes empty. We prove
that the average run length is at least $1/\alpha$, resulting in a change
detection scheme with minimal structural assumptions~(thus allowing for
possibly dependent observations, and nonparametric distribution classes), but
strong guarantees. Our approach bears an interesting parallel with the
reduction from change detection to sequential testing of Lorden (1971) and the
e-detector of Shin et al. (2022).

通过使用置信序列，我们将时序变化检测简化为时序估计，并在所有活动置信序列的交集为空时宣布变化。我们证明了平均运行长度至少为 1/α，从而实现了具有最小结构假设但具有强有力保证的变化检测方案。

将顺序变化检测简化为顺序估计

Reducing sequential change detection to sequential estimation

Deep learning has recently gained immense popularity in the Earth sciences as
it enables us to formulate purely data-driven models of complex Earth system
processes. Deep learning-based weather prediction (DLWP) models have made
significant progress in the last few years, achieving forecast skills
comparable to established numerical weather prediction (NWP) models with
comparatively lesser computational costs. In order to train accurate, reliable,
and tractable DLWP models with several millions of parameters, the model design
needs to incorporate suitable inductive biases that encode structural
assumptions about the data and modelled processes. When chosen appropriately,
these biases enable faster learning and better generalisation to unseen data.
Although inductive biases play a crucial role in successful DLWP models, they
are often not stated explicitly and how they contribute to model performance
remains unclear. Here, we review and analyse the inductive biases of six
state-of-the-art DLWP models, involving a deeper look at five key design
elements: input data, forecasting objective, loss components, layered design of
the deep learning architectures, and optimisation methods. We show how the
design choices made in each of the five design elements relate to structural
assumptions. Given recent developments in the broader DL community, we
anticipate that the future of DLWP will likely see a wider use of foundation
models -- large models pre-trained on big databases with self-supervised
learning -- combined with explicit physics-informed inductive biases that allow
the models to provide competitive forecasts even at the more challenging
subseasonal-to-seasonal scales.

分析了深度学习天气预测模型的感知偏差及设计元素，包括输入数据、预测目标、损失组成、神经网络结构和优化方法，并指出未来的发展趋势。

深度学习模型中的归纳偏差在天气预测中的应用

Inductive biases in deep learning models for weather prediction

Traditional language models treat language as a finite state automaton on a
probability space over words. This is a very strong assumption when modeling
something inherently complex such as language. In this paper, we challenge this
by showing how the linear chain assumption inherent in previous work can be
translated into a sequential composition tree. We then propose a new model that
marginalizes over all possible composition trees thereby removing any
underlying structural assumptions. As the partition function of this new model
is intractable, we use a recently proposed sentence level evaluation metric
Contrastive Entropy to evaluate our model. Given this new evaluation metric, we
report more than 100% improvement across distortion levels over current state
of the art recurrent neural network based language models.

本文提出了一种新的语言模型，通过序列组合树来取代之前基于线性链的假设，消除了结构上的任何假设，利用对比熵作为评估度量标准对新模型进行评估，相较于以往基于递归神经网络的模型，在失真级别上获得了超过 100% 的改进。