Document layout analysis is a key area in document research, involving
techniques like text mining and visual analysis. Despite various methods
developed to tackle layout analysis, a critical but frequently overlooked
problem is the scarcity of labeled data needed for analyses. With the rise of
internet use, an overwhelming number of documents are now available online,
making the process of accurately labeling them for research purposes
increasingly challenging and labor-intensive. Moreover, the diversity of
documents online presents a unique set of challenges in maintaining the quality
and consistency of these labels, further complicating document layout analysis
in the digital era. To address this, we employ a vision-based approach for
analyzing document layouts designed to train a network without labels. Instead,
we focus on pre-training, initially generating simple object masks from the
unlabeled document images. These masks are then used to train a detector,
enhancing object detection and segmentation performance. The model's
effectiveness is further amplified through several unsupervised training
iterations, continuously refining its performance. This approach significantly
advances document layout analysis, particularly precision and efficiency,
without labels.

通过基于视觉的方法，本研究利用未标记的网络文档图像生成简单的对象掩码，通过无监督训练迭代持续改进该模型的性能，从而显著提高文档布局分析的准确性和效率。

无监督文档布局分析

UnSupDLA: Towards Unsupervised Document Layout Analysis

The recent success of large pre-trained language models (PLMs) heavily hinges
on massive labeled data, which typically produces inferior performance in
low-resource scenarios. To remedy this dilemma, we study self-training as one
of the predominant semi-supervised learning (SSL) approaches, which utilizes
large-scale unlabeled data to generate synthetic examples. However, too many
noisy labels will hurt the model performance, and the self-training procedure
requires multiple training iterations making it more expensive if all the model
parameters of the PLM are updated. This paper presents UPET, a novel
Uncertainty-aware Parameter-Efficient self-Training framework to effectively
and efficiently address the labeled data scarcity issue. Specifically, we
incorporate Monte Carlo (MC) dropout in Bayesian neural network (BNN) to
perform uncertainty estimation for the teacher model and then judiciously
select reliable pseudo-labeled examples based on confidence and certainty.
During the student training, we introduce multiple parameter-efficient learning
(PEL) paradigms that allow the optimization of only a small percentage of
parameters. We also propose a novel Easy-Hard Contrastive Tuning to enhance the
robustness and generalization. Extensive experiments over multiple downstream
tasks demonstrate that UPET achieves a substantial improvement in terms of
performance and efficiency. Our codes and data are released at https:
//github.com/wjn1996/UPET.

UPET 是一个新颖的、针对标记数据稀缺问题的不确定性感知的参数高效自训练框架，通过在蒙特卡洛（MC）dropout 和贝叶斯神经网络（BNN）中进行不确定性估计，并基于置信度和确定性选择可靠的伪标记样本，在学生训练中引入多个参数高效学习（PEL）范式来仅优化一小部分参数，以及提出了一种新颖的 Easy-Hard 对比调优方法来增强鲁棒性和泛化性能。大量的实验证明 UPET 在性能和效率方面取得了显著改进。

半监督语言理解中基于不确定性的参数高效自训练

Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised  Language Understanding

Adversarial robustness is a research area that has recently received a lot of
attention in the quest for trustworthy artificial intelligence. However, recent
works on adversarial robustness have focused on supervised learning where it is
assumed that labeled data is plentiful. In this paper, we investigate
semi-supervised adversarial training where labeled data is scarce. We derive
two upper bounds for the robust risk and propose a regularization term for
unlabeled data motivated by these two upper bounds. Then, we develop a
semi-supervised adversarial training algorithm that combines the proposed
regularization term with knowledge distillation using a semi-supervised teacher
(i.e., a teacher model trained using a semi-supervised learning algorithm). Our
experiments show that our proposed algorithm achieves state-of-the-art
performance with significant margins compared to existing algorithms. In
particular, compared to supervised learning algorithms, performance of our
proposed algorithm is not much worse even when the amount of labeled data is
very small. For example, our algorithm with only 8\% labeled data is comparable
to supervised adversarial training algorithms that use all labeled data, both
in terms of standard and robust accuracies on CIFAR-10.

关键词：对抗鲁棒性、半监督对抗训练、有标签数据稀缺、正则化项、知识蒸馏。摘要：本文研究了对抗鲁棒性的半监督对抗训练，探究了稀缺有标签数据情况下的两个鲁棒风险上界，并提出了正则化项以应用于无标签数据，最后通过半监督教师模型的知识蒸馏结合提出的正则化项进行半监督对抗训练算法。实验证明，相较于现有算法，我们提出的算法在性能方面取得了显著的提升，并且即使有限的有标签数据量，我们的算法仍能与使用所有标签数据的监督对抗训练算法相媲美，无论是在标准准确率还是在鲁棒准确率上，比如我们的算法只使用 8％的有标签数据时，在 CIFAR-10 数据集上与监督对抗训练算法相比表现接近。