Can AI help automate human-easy but computer-hard data preparation tasks that
burden data scientists, practitioners, and crowd workers? We answer this
question by presenting RPT, a denoising auto-encoder for tuple-to-X models (X
could be tuple, token, label, JSON, and so on). RPT is pre-trained for a
tuple-to-tuple model by corrupting the input tuple and then learning a model to
reconstruct the original tuple. It adopts a Transformer-based neural
translation architecture that consists of a bidirectional encoder (similar to
BERT) and a left-to-right autoregressive decoder (similar to GPT), leading to a
generalization of both BERT and GPT. The pre-trained RPT can already support
several common data preparation tasks such as data cleaning, auto-completion
and schema matching. Better still, RPT can be fine-tuned on a wide range of
data preparation tasks, such as value normalization, data transformation, data
annotation, etc. To complement RPT, we also discuss several appealing
techniques such as collaborative training and few-shot learning for entity
resolution, and few-shot learning and NLP question-answering for information
extraction. In addition, we identify a series of research opportunities to
advance the field of data preparation.

RPT 是一种加噪自编码器，可用于元组到元组模型的预训练。它采用了一种基于 Transformer 的神经翻译架构，可推广 BERT 和 GPT，并可支持数据准备任务，例如数据清理，值规范化，数据变换和数据注释等。此外，它具有 fine-tune 和 few-shot learning 等技术，并提出了一系列研究机遇，以推动数据准备领域的发展。