Automatic document summarization aims to produce a concise summary covering
the input document's salient information. Within a report document, the salient
information can be scattered in the textual and non-textual content. However,
existing document summarization datasets and methods usually focus on the text
and filter out the non-textual content. Missing tabular data can limit produced
summaries' informativeness, especially when summaries require covering
quantitative descriptions of critical metrics in tables. Existing datasets and
methods cannot meet the requirements of summarizing long text and multiple
tables in each report. To deal with the scarcity of available data, we propose
FINDSum, the first large-scale dataset for long text and multi-table
summarization. Built on 21,125 annual reports from 3,794 companies, it has two
subsets for summarizing each company's results of operations and liquidity. To
summarize the long text and dozens of tables in each report, we present three
types of summarization methods. Besides, we propose a set of evaluation metrics
to assess the usage of numerical information in produced summaries. Dataset
analyses and experimental results indicate the importance of jointly
considering input textual and tabular data when summarizing report documents.

旨在创造简明摘要以涵盖输入文档的要点信息。自动文档摘要对文本和非文本内容进行了讨论，提出了涵盖定量描述的关键指标的表格数据的重要性，提出了一个大规模数据集和三种摘要方法，还评估了评估指标

长文本和多表总结：数据集和方法

Long Text and Multi-Table Summarization: Dataset and Method

Document-level relation extraction (DocRE) aims to identify semantic labels
among entities within a single document. One major challenge of DocRE is to dig
decisive details regarding a specific entity pair from long text. However, in
many cases, only a fraction of text carries required information, even in the
manually labeled supporting evidence. To better capture and exploit instructive
information, we propose a novel expLicit syntAx Refinement and Subsentence
mOdeliNg based framework (LARSON). By introducing extra syntactic information,
LARSON can model subsentences of arbitrary granularity and efficiently screen
instructive ones. Moreover, we incorporate refined syntax into text
representations which further improves the performance of LARSON. Experimental
results on three benchmark datasets (DocRED, CDR, and GDA) demonstrate that
LARSON significantly outperforms existing methods.

该研究提出了一种新的基于显式句法改进和子句建模的文档层面关系抽取框架（LARSON），该框架旨在更好地捕获和利用指导信息，进而提高文档内实体之间语义关系的准确性。实验结果表明，LARSON 在三个基准数据集上的性能优于现有方法。

利用显式语法优化和子句建模提升文档级关系抽取

Not Just Plain Text! Fuel Document-Level Relation Extraction with Explicit Syntax Refinement and Subsentence Modeling

BERT is a widely used pre-trained model in natural language processing.
However, since BERT is quadratic to the text length, the BERT model is
difficult to be used directly on the long-text corpus. In some fields, the
collected text data may be quite long, such as in the health care field.
Therefore, to apply the pre-trained language knowledge of BERT to long text, in
this paper, imitating the skimming-intensive reading method used by humans when
reading a long paragraph, the Skimming-Intensive Model (SkIn) is proposed. It
can dynamically select the critical information in the text so that the
sentence input into the BERT-Base model is significantly shortened, which can
effectively save the cost of the classification algorithm. Experiments show
that the SkIn method has achieved superior accuracy than the baselines on
long-text classification datasets in the medical field, while its time and
space requirements increase linearly with the text length, alleviating the time
and space overflow problem of basic BERT on long-text data.

本文提出了一种名为 Skimming-Intensive Model （SkIn）的方法，通过模拟人类阅读长段落时的高效阅读方法，动态选择文本中的关键信息，从而缩短输入 BERT 模型的句子长度，有效降低分类算法的成本，该方法在医疗领域的长文本分类数据集上已经取得了比基线更优秀的准确性，同时其时间和空间要求随着文本长度的增加呈线性增长，缓解了基本 BERT 在长文本数据上的时间和空间溢出问题。

SkIn: 使用 BERT 进行文本分类的医学语料库的快速筛选

SkIn: Skimming-Intensive Long-Text Classification Using BERT for Medical Corpus

The Transformer model is widely successful on many natural language
processing tasks. However, the quadratic complexity of self-attention limit its
application on long text. In this paper, adopting a fine-to-coarse attention
mechanism on multi-scale spans via binary partitioning (BP), we propose
BP-Transformer (BPT for short). BPT yields $O(k\cdot n\log (n/k))$ connections
where $k$ is a hyperparameter to control the density of attention. BPT has a
good balance between computation complexity and model capacity. A series of
experiments on text classification, machine translation and language modeling
shows BPT has a superior performance for long text than previous self-attention
models. Our code, hyperparameters and CUDA kernels for sparse attention are
available in PyTorch.

本文提出了一种基于多尺度跨度和二进制分多方式的 BP-Transformer 模型，该模型在处理长文本时性能优于之前的自注意力模型。