Empirical methods in geoparsing have thus far lacked a standard evaluation framework described as the task, data and metrics used to establish state-of-the-art systems. Evaluation is further made inconsistent, even unrepresentative of real-world usage, by the lack of distinction between the different types of toponyms, which necessitates new guidelines, a consolidation of metrics and a detailed toponym taxonomy with implications for Named Entity Recognition (NER). To address these deficiencies, our manuscript introduces such framework in three parts. Part 1) Task Definition: clarified via corpus linguistic analysis proposing a fine-grained Pragmatic Taxonomy of Toponyms with new guidelines. Part 2) Evaluation Data: shared via a dataset called GeoWebNews to provide test/train data to enable immediate use of our contributions. In addition to fine-grained Geotagging and Toponym Resolution (Geocoding), this dataset is also suitable for prototyping machine learning NLP models. Part 3) Metrics: discussed and reviewed for a rigorous evaluation with appropriate recommendations for NER/Geoparsing practitioners.

该研究提出了一个三部分新框架，包括任务定义、指标和评估数据，即通过语料库语言分析提出一个细粒度的地名实用分类学，讨论和审查任务的指标，并共享一个新的数据集GeoWebNews以提供测试/培训示例和启用我们贡献的即时使用； 除了细粒度Geotagging和Toponym Resolution（地理编码），该数据集也适用于机器学习NLP模型的原型设计和评估

地理解析评估实践指南