In the contemporary data-driven landscape, ensuring data quality (DQ) is
crucial for deriving actionable insights from vast data repositories. The
objective of this study is to explore the potential for automating data quality
management within data warehouses as data repository commonly used by large
organizations. By conducting a systematic review of existing DQ tools available
in the market and academic literature, the study assesses their capability to
automatically detect and enforce data quality rules. The review encompassed 151
tools from various sources, revealing that most current tools focus on data
cleansing and fixing in domain-specific databases rather than data warehouses.
Only a limited number of tools, specifically ten, demonstrated the capability
to detect DQ rules, not to mention implementing this in data warehouses. The
findings underscore a significant gap in the market and academic research
regarding AI-augmented DQ rule detection in data warehouses. This paper
advocates for further development in this area to enhance the efficiency of DQ
management processes, reduce human workload, and lower costs. The study
highlights the necessity of advanced tools for automated DQ rule detection,
paving the way for improved practices in data quality management tailored to
data warehouse environments. The study can guide organizations in selecting
data quality tool that would meet their requirements most.

该研究旨在探索自动化数据质量管理的潜力，以提高数据质量检测规则在数据仓库中的实施，进而增强数据质量管理的效率。

朝着增强的数据质量管理：数据仓库中数据质量规则定义的自动化

Towards augmented data quality management: Automation of Data Quality  Rule Definition in Data Warehouses

Hydra is a system which utilizes computer vision to perform near real time
data quality management, initially developed for Hall-D in 2019. Since then, it
has been deployed across all experimental halls at Jefferson Lab, with the
CLAS12 collaboration in Hall-B being the first outside of GlueX to fully
utilize Hydra. The system comprises back end processes that manage the models,
their inferences, and the data flow. The front-end components, accessible via
web pages, allow detector experts and shift crews to view and interact with the
system. This talk will give an overview of the Hydra system as well as
highlight significant developments in Hydra's feature set, acute challenges
with operating Hydra in all halls, and lessons learned along the way.

Hydra 是一种利用计算机视觉进行近实时数据质量管理的系统，最初于 2019 年为 Hall-D 开发。自那时以来，它已经在 Jefferson Lab 的所有实验大厅部署，其中 Hall-B 的 CLAS12 合作团队是继 GlueX 之后首个完全利用 Hydra 的系统。该系统包括后端处理过程，负责管理模型、推断和数据流程。通过网页访问的前端组件允许探测器专家和班组与系统进行交互。本文将概述 Hydra 系统，并突出 Hydra 特性集的重要发展、在所有大厅中操作 Hydra 的急迫挑战以及过程中吸取到的经验教训。