Cardinality estimation methods based on probability distribution estimation have achieved high-precision estimation results compared to traditional methods. However, the most advanced methods suffer from high estimation costs due to the sampling method they use when dealing with range queries. Also, such a sampling method makes them difficult to differentiate, so the supervision signal from the query workload is difficult to train the model to improve the accuracy of cardinality estimation. In this paper, we propose a new hybrid and deterministic modeling approach (Duet) for the cardinality estimation problem which has better efficiency and scalability compared to previous approaches. Duet allows for direct cardinality estimation of range queries with significantly lower time and memory costs, as well as in a differentiable form. As the prediction process of this approach is differentiable, we can incorporate queries with larger model estimation errors into the training process to address the long-tail distribution problem of model estimation errors on high dimensional tables. We evaluate Duet on classical datasets and benchmarks, and the results prove the effectiveness of Duet.

我们提出了一种名为Duet的新型混合确定建模方法，用于基数估计问题，与之前的方法相比，具有更好的效率和可扩展性。Duet允许以可微分的形式直接估计范围查询的基数，时间和内存成本大大降低。通过将模型估计错误较大的查询纳入训练过程中，我们可以解决高维表上模型估计误差的长尾分布问题。我们对经典数据集和基准进行了Duet的评估，结果证明了Duet的有效性。

Duet: 高效且可扩展的混合神经关联理解