Pioneering data profiling systems such as Metanome and OpenClean brought
public attention to science-intensive data profiling. This type of profiling
aims to extract complex patterns (primitives) such as functional dependencies,
data constraints, association rules, and others. However, these tools are
research prototypes rather than production-ready systems.
The following work presents Desbordante - a high-performance
science-intensive data profiler with open source code. Unlike similar systems,
it is built with emphasis on industrial application in a multi-user
environment. It is efficient, resilient to crashes, and scalable. Its
efficiency is ensured by implementing discovery algorithms in C++, resilience
is achieved by extensive use of containerization, and scalability is based on
replication of containers.
Desbordante aims to open industrial-grade primitive discovery to a broader
public, focusing on domain experts who are not IT professionals. Aside from the
discovery of various primitives, Desbordante offers primitive validation, which
not only reports whether a given instance of primitive holds or not, but also
points out what prevents it from holding via the use of special screens. Next,
Desbordante supports pipelines - ready-to-use functionality implemented using
the discovered primitives, for example, typo detection. We provide built-in
pipelines, and the users can construct their own via provided Python bindings.
Unlike other profilers, Desbordante works not only with tabular data, but with
graph and transactional data as well.
In this paper, we present Desbordante, the vision behind it and its
use-cases. To provide a more in-depth perspective, we discuss its current
state, architecture, and design decisions it is built on. Additionally, we
outline our future plans.

该研究提出了 Desbordante—— 一种基于工业应用的高性能数据分析系统，旨在为广大科学领域的专家揭示出数据中的复杂模式，并提供验证及流水线等功能，同时支持对表格数据、图形数据和交易数据的处理。