In the pursuit of robust and generalizable environment perception and
language understanding, the ubiquitous challenge of dataset bias continues to
plague vision-and-language navigation (VLN) agents, hindering their performance
in unseen environments. This paper introduces the generalized cross-modal
causal transformer (GOAT), a pioneering solution rooted in the paradigm of
causal inference. By delving into both observable and unobservable confounders
within vision, language, and history, we propose the back-door and front-door
adjustment causal learning (BACL and FACL) modules to promote unbiased learning
by comprehensively mitigating potential spurious correlations. Additionally, to
capture global confounder features, we propose a cross-modal feature pooling
(CFP) module supervised by contrastive learning, which is also shown to be
effective in improving cross-modal representations during pre-training.
Extensive experiments across multiple VLN datasets (R2R, REVERIE, RxR, and
SOON) underscore the superiority of our proposed method over previous
state-of-the-art approaches. Code is available at
this https URL

通过基于因果推断范式的通用交叉模态因果变换器（GOAT）解决视觉语言导航（VLN）代理程序在未知环境中性能受限的数据集偏差问题，提出通过背门调整和前门调整因果学习模块来全面减轻潜在的伪相关性并提升无偏学习，同时通过对比学习监督的交叉模态特征汇聚模块（CFP）来捕捉全局的混杂物特征，实验证明我们的方法在多个 VLN 数据集上优于先前的最先进方法。

通过因果学习的视觉语言导航

Vision-and-Language Navigation via Causal Learning

Estimating counterfactual outcomes over time from observational data is
relevant for many applications (e.g., personalized medicine). Yet,
state-of-the-art methods build upon simple long short-term memory (LSTM)
networks, thus rendering inferences for complex, long-range dependencies
challenging. In this paper, we develop a novel Causal Transformer for
estimating counterfactual outcomes over time. Our model is specifically
designed to capture complex, long-range dependencies among time-varying
confounders. For this, we combine three transformer subnetworks with separate
inputs for time-varying covariates, previous treatments, and previous outcomes
into a joint network with in-between cross-attentions. We further develop a
custom, end-to-end training procedure for our Causal Transformer. Specifically,
we propose a novel counterfactual domain confusion loss to address confounding
bias: it aims to learn adversarial balanced representations, so that they are
predictive of the next outcome but non-predictive of the current treatment
assignment. We evaluate our Causal Transformer based on synthetic and
real-world datasets, where it achieves superior performance over current
baselines. To the best of our knowledge, this is the first work proposing
transformer-based architecture for estimating counterfactual outcomes from
longitudinal data.

本文提出了一种基于 Transformer 的因果模型，通过特定设计的模型和训练方法，能够准确地对复杂、长期相关和混淆因素进行因果推理并估算出时间序列的反事实结果。