This paper presents a novel benchmarking framework Dyport for evaluating
biomedical hypothesis generation systems. Utilizing curated datasets, our
approach tests these systems under realistic conditions, enhancing the
relevance of our evaluations. We integrate knowledge from the curated databases
into a dynamic graph, accompanied by a method to quantify discovery importance.
This not only assesses hypothesis accuracy but also their potential impact in
biomedical research which significantly extends traditional link prediction
benchmarks. Applicability of our benchmarking process is demonstrated on
several link prediction systems applied on biomedical semantic knowledge
graphs. Being flexible, our benchmarking system is designed for broad
application in hypothesis generation quality verification, aiming to expand the
scope of scientific discovery within the biomedical research community.
Availability and implementation: Dyport framework is fully open-source. All
code and datasets are available at: this https URL

该研究提出了一个新颖的 benchmarking 框架 Dyport，用于评估生物医学假设生成系统。利用经过筛选的数据集，在真实条件下测试这些系统，增强了我们评估的相关性。我们将筛选数据库中的知识整合到动态图中，并伴随着一种量化发现重要性的方法。这不仅评估了假设的准确性，还评估了它们在生物医学研究中的潜在影响，这显著扩展了传统的链接预测基准。我们的 benchmarking 过程的适用性在应用于生物医学语义知识图上的几个链接预测系统上得到了证明。我们的灵活性 benchmarking 系统旨在广泛应用于假设生成质量验证，旨在扩大生物医学研究界的科学发现范围。Dyport 框架完全开源，所有代码和数据集都可以在此 https 的 URL 中获得。