In the context of medical decision making, counterfactual prediction enables
clinicians to predict treatment outcomes of interest under alternative courses
of therapeutic actions given observed patient history. Prior machine learning
approaches for counterfactual predictions under time-varying treatments focus
on static time-varying treatment regimes where treatments do not depend on
previous covariate history. In this work, we present G-Transformer, a
Transformer-based framework supporting g-computation for counterfactual
prediction under dynamic and time-varying treatment strategies. G-Transfomer
captures complex, long-range dependencies in time-varying covariates using a
Transformer architecture. G-Transformer estimates the conditional distribution
of relevant covariates given covariate and treatment history at each time point
using an encoder architecture, then produces Monte Carlo estimates of
counterfactual outcomes by simulating forward patient trajectories under
treatment strategies of interest. We evaluate G-Transformer extensively using
two simulated longitudinal datasets from mechanistic models, and a real-world
sepsis ICU dataset from MIMIC-IV. G-Transformer outperforms both classical and
state-of-the-art counterfactual prediction models in these settings. To the
best of our knowledge, this is the first Transformer-based architecture for
counterfactual outcome prediction under dynamic and time-varying treatment
strategies. Code will be released upon publication of the paper.

基于 G-Transformer 框架的对动态和时间变化的治疗策略进行反事实预测的研究中，通过模拟患者在感兴趣的治疗策略下的前进轨迹，使用 Transformer 架构捕获时间变化的协变量的复杂长程依赖并评估该框架的性能。

G-Transformer：动态和时变治疗制度下的反事实结果预测

G-Transformer: Counterfactual Outcome Prediction under Dynamic and  Time-varying Treatment Regimes

Estimating potential outcomes for treatments over time based on observational
data is important for personalized decision-making in medicine. Yet, existing
neural methods for this task suffer from either (a) bias or (b) large variance.
In order to address both limitations, we introduce the G-transformer (GT). Our
GT is a novel, neural end-to-end model designed for unbiased, low-variance
estimation of conditional average potential outcomes (CAPOs) over time.
Specifically, our GT is the first neural model to perform regression-based
iterative G-computation for CAPOs in the time-varying setting. We evaluate the
effectiveness of our GT across various experiments. In sum, this work
represents a significant step towards personalized decision-making from
electronic health records.

给医学个性化决策做出重要贡献的研究论文，提出了一种新的神经网络模型 G-transformer，用于基于观察数据对治疗效果的时间估计，该模型能够准确、低方差地预测条件期望效果，具有迭代回归和处理电子健康档案的能力。

G-Transformer 用于时间上的条件平均潜在结果估计

G-Transformer for Conditional Average Potential Outcome Estimation over  Time

Document-level MT models are still far from satisfactory. Existing work
extend translation unit from single sentence to multiple sentences. However,
study shows that when we further enlarge the translation unit to a whole
document, supervised training of Transformer can fail. In this paper, we find
such failure is not caused by overfitting, but by sticking around local minima
during training. Our analysis shows that the increased complexity of
target-to-source attention is a reason for the failure. As a solution, we
propose G-Transformer, introducing locality assumption as an inductive bias
into Transformer, reducing the hypothesis space of the attention from target to
source. Experiments show that G-Transformer converges faster and more stably
than Transformer, achieving new state-of-the-art BLEU scores for both
non-pretraining and pre-training settings on three benchmark datasets.

通过引入归纳偏置（locality assumption）作为一种假设，从目标到源的注意力的假设空间减小，来解决翻译单元扩展至整个文档时出现的训练失败问题，我们提出了 G-Transformer。实验表明，G-Transformer 比 Transformer 更快、更稳定地收敛，在三个基准数据集上实现了新的最优 BLEU 分数。