In this paper, we introduce strategies for developing private Key Information
Extraction (KIE) systems by leveraging large pretrained document foundation
models in conjunction with differential privacy (DP), federated learning (FL),
and Differentially Private Federated Learning (DP-FL). Through extensive
experimentation on six benchmark datasets (FUNSD, CORD, SROIE, WildReceipts,
XFUND, and DOCILE), we demonstrate that large document foundation models can be
effectively fine-tuned for the KIE task under private settings to achieve
adequate performance while maintaining strong privacy guarantees. Moreover, by
thoroughly analyzing the impact of various training and model parameters on
model performance, we propose simple yet effective guidelines for achieving an
optimal privacy-utility trade-off for the KIE task under global DP. Finally, we
introduce FeAm-DP, a novel DP-FL algorithm that enables efficiently upscaling
global DP from a standalone context to a multi-client federated environment. We
conduct a comprehensive evaluation of the algorithm across various client and
privacy settings, and demonstrate its capability to achieve comparable
performance and privacy guarantees to standalone DP, even when accommodating an
increasing number of participating clients. Overall, our study offers valuable
insights into the development of private KIE systems, and highlights the
potential of document foundation models for privacy-preserved Document AI
applications. To the best of authors' knowledge, this is the first work that
explores privacy preserved document KIE using document foundation models.

通过差分隐私、联邦学习和差分隐私联邦学习的大型预训练文档基础模型，我们介绍了开发私有键信息提取（KIE）系统的策略。通过在六个基准数据集上的广泛实验，我们证明了大型文档基础模型可以在私有环境下进行有效微调，以实现足够的性能并保持强大的隐私保证。最后，我们引入了一种名为 FeAm-DP 的新型 DP-FL 算法，它能够将全局差分隐私从独立环境有效地扩展到多客户联邦环境。总体而言，我们的研究为私有 KIE 系统的开发提供了宝贵的见解，并凸显了文档基础模型在隐私保护文档 AI 应用中的潜力。