We present a detailed replication study of the BASS framework, an abstractive
summarization system based on the notion of Unified Semantic Graphs. Our
investigation includes challenges in replicating key components and an ablation
study to systematically isolate error sources rooted in replicating novel
components. Our findings reveal discrepancies in performance compared to the
original work. We highlight the significance of paying careful attention even
to reasonably omitted details for replicating advanced frameworks like BASS,
and emphasize key practices for writing replicable papers.

详细复制研究 BASS 框架，一种基于统一语义图的抽象摘要系统，包括复制关键组件的挑战和消除源于复制新组件的错误来源的消融研究。我们的研究结果显示与原论文相比性能存在差异，强调即使是被合理省略的细节也需要仔细关注，以便复制先进的框架，同时强调编写可复制论文的关键实践。

对 BASS 的再次审视：利用统一语义图提升抽象摘要生成的研究复制

A Second Look on BASS -- Boosting Abstractive Summarization with Unified  Semantic Graphs -- A Replication Study

Text retrieval using learned dense representations has recently emerged as a
promising alternative to "traditional" text retrieval using sparse bag-of-words
representations. One recent work that has garnered much attention is the dense
passage retriever (DPR) technique proposed by Karpukhin et al. (2020) for
end-to-end open-domain question answering. We present a replication study of
this work, starting with model checkpoints provided by the authors, but
otherwise from an independent implementation in our group's Pyserini IR toolkit
and PyGaggle neural text ranking library. Although our experimental results
largely verify the claims of the original paper, we arrived at two important
additional findings that contribute to a better understanding of DPR: First, it
appears that the original authors under-report the effectiveness of the BM25
baseline and hence also dense--sparse hybrid retrieval results. Second, by
incorporating evidence from the retriever and an improved answer span scoring
technique, we are able to improve end-to-end question answering effectiveness
using exactly the same models as in the original work.

本研究对 Karpukhin 等人于 2020 年提出的基于稠密编码的全域问答模型（DPR）进行了复制研究，其中发现原作者低估了 BM25 基准检索的有效性。我们通过改进的答案跨度评分技术，使用与原论文相同的模型获得更好的端到端问题答案准确性。

稠密通道检索器的复制研究

A Replication Study of Dense Passage Retriever

Language model pretraining has led to significant performance gains but
careful comparison between different approaches is challenging. Training is
computationally expensive, often done on private datasets of different sizes,
and, as we will show, hyperparameter choices have significant impact on the
final results. We present a replication study of BERT pretraining (Devlin et
al., 2019) that carefully measures the impact of many key hyperparameters and
training data size. We find that BERT was significantly undertrained, and can
match or exceed the performance of every model published after it. Our best
model achieves state-of-the-art results on GLUE, RACE and SQuAD. These results
highlight the importance of previously overlooked design choices, and raise
questions about the source of recently reported improvements. We release our
models and code.

本研究进行了 BERT 预训练的复现研究，并对其关键的超参数和训练数据大小的影响进行了测量，发现 BERT 被显著低估，并且可以超过其之后发布的每个模型的表现，提高了以前被忽视的设计选择的重要性，并引发了对最近报道的改进的疑问。