We present CausalSim, a causal inference framework for unbiased trace-driven
simulation. Current trace-driven simulators assume that the interventions being
simulated (e.g., a new algorithm) would not affect the validity of the traces.
However, real-world traces are often biased by the choices of algorithms made
during trace collection, and hence replaying traces under an intervention may
lead to incorrect results. CausalSim addresses this challenge by learning a
causal model of the system dynamics and latent factors capturing the underlying
system conditions during trace collection. It learns these models using an
initial randomized control trial (RCT) under a fixed set of algorithms, and
then applies them to remove biases from trace data when simulating new
algorithms.
Key to CausalSim is mapping unbiased trace-driven simulation to a tensor
completion problem with extremely sparse observations. By exploiting a basic
distributional invariance property present in RCT data, CausalSim enables a
novel tensor completion method despite the sparsity of observations. Our
extensive evaluation of CausalSim on both real and synthetic datasets,
including more than ten months of real data from the Puffer video streaming
system show it improves simulation accuracy, reducing errors by 53% and 61% on
average compared to expert-designed and supervised learning baselines.
Moreover, CausalSim provides markedly different insights about ABR algorithms
compared to the biased baseline simulator, which we validate with a real
deployment.

CausalSim 是一个建立在因果推断框架之上的模拟器，其使用一个基于张量补全问题的方法去消除由于算法选择导致的偏差，实验表明使用 CausalSim 进行模拟的结果比专家设计和监督学习的基准结果平均下降了 53% 至 61%。