The paper describes neural models developed for the DAGPap22 shared task
hosted at the Third Workshop on Scholarly Document Processing. This shared task
targets the automatic detection of generated scientific papers. Our work
focuses on comparing different transformer-based models as well as using
additional datasets and techniques to deal with imbalanced classes. As a final
submission, we utilized an ensemble of SciBERT, RoBERTa, and DeBERTa fine-tuned
using random oversampling technique. Our model achieved 99.24% in terms of
F1-score. The official evaluation results have put our system at the third
place.

该论文讨论了在第三届学术文档处理研讨会上开展的 DAGPap22 共享任务中开发的神经模型，侧重于比较不同的基于 transformer 的模型并使用附加数据集和技术来处理不平衡的类别，在提交最终版本时，我们使用经过随机过采样技术微调的 SciBERT，RoBERTa 和 DeBERTa 集成模型达到了 99.24% 的 F1 得分，官方评估结果将我们的系统排在第三位。

利用 Transformer 模型集合检测生成的科学论文

Detecting Generated Scientific Papers using an Ensemble of Transformer Models

Score-based generative models (SGMs) are a recent breakthrough in generating
fake images. SGMs are known to surpass other generative models, e.g.,
generative adversarial networks (GANs) and variational autoencoders (VAEs).
Being inspired by their big success, in this work, we fully customize them for
generating fake tabular data. In particular, we are interested in oversampling
minor classes since imbalanced classes frequently lead to sub-optimal training
outcomes. To our knowledge, we are the first presenting a score-based tabular
data oversampling method. Firstly, we re-design our own score network since we
have to process tabular data. Secondly, we propose two options for our
generation method: the former is equivalent to a style transfer for tabular
data and the latter uses the standard generative policy of SGMs. Lastly, we
define a fine-tuning method, which further enhances the oversampling quality.
In our experiments with 6 datasets and 10 baselines, our method outperforms
other oversampling methods in all cases.

本论文在以往 score-based generative models 成功生成虚假图像的基础上，自主研发了在 tabular data 数据集上使用的 score-based 过采样方法，该方法采用自有的 score network，提出了两种生成方法，进一步使用 fine-tuning 方法改进，实验证明在 6 个数据集上，该方法均能优于其他过采样方法。

SOS：针对表格数据的基于得分的过采样

SOS: Score-based Oversampling for Tabular Data

In this paper, we propose a new metric to measure goodness-of-fit for
classifiers, the Real World Cost function. This metric factors in information
about a real world problem, such as financial impact, that other measures like
accuracy or F1 do not. This metric is also more directly interpretable for
users. To optimize for this metric, we introduce the Real-World- Weight
Crossentropy loss function, in both binary and single-label classification
variants. Both variants allow direct input of real world costs as weights. For
single-label, multicategory classification, our loss function also allows
direct penalization of probabilistic false positives, weighted by label, during
the training of a machine learning model. We compare the design of our loss
function to the binary crossentropy and categorical crossentropy functions, as
well as their weighted variants, to discuss the potential for improvement in
handling a variety of known shortcomings of machine learning, ranging from
imbalanced classes to medical diagnostic error to reinforcement of social bias.
We create scenarios that emulate those issues using the MNIST data set and
demonstrate empirical results of our new loss function. Finally, we sketch a
proof of this function based on Maximum Likelihood Estimation and discuss
future directions.

该研究提出了一种新的度量分类器拟合度的度量标准 —— 实际成本函数，该函数结合了有关实际问题的信息，例如财务影响，以及用户更易于解释的特性。为了优化这个度量标准，作者介绍了基于实际成本函数的损失函数，并比较了损失函数的设计和已有的损失函数，通过 MNIST 数据集的模拟情境对其效果进行了实证研究。