Large language models (LLMs) and multimodal large language models (MLLMs)
have shown excellent general capabilities, even exhibiting adaptability in many
professional domains such as law, economics, transportation, and medicine.
Currently, many domain-specific benchmarks have been proposed to verify the
performance of (M)LLMs in specific fields. Among various domains,
transportation plays a crucial role in modern society as it impacts the
economy, the environment, and the quality of life for billions of people.
However, it is unclear how much traffic knowledge (M)LLMs possess and whether
they can reliably perform transportation-related tasks. To address this gap, we
propose TransportationGames, a carefully designed and thorough evaluation
benchmark for assessing (M)LLMs in the transportation domain. By
comprehensively considering the applications in real-world scenarios and
referring to the first three levels in Bloom's Taxonomy, we test the
performance of various (M)LLMs in memorizing, understanding, and applying
transportation knowledge by the selected tasks. The experimental results show
that although some models perform well in some tasks, there is still much room
for improvement overall. We hope the release of TransportationGames can serve
as a foundation for future research, thereby accelerating the implementation
and application of (M)LLMs in the transportation domain.

我们提出了 TransportationGames，这是一个经过精心设计和全面评估的基准测试，用于评估 (多模态) 大规模语言模型在交通领域中的性能，并发现尽管某些模型在某些任务中表现良好，但整体上仍有很大的改进空间。我们希望 TransportationGames 的发布能够成为未来研究的基础，加速 (多模态) 大规模语言模型在交通领域中的实施和应用。

交通游戏：对（多模式）大型语言模型的交通知识进行基准评估

TransportationGames: Benchmarking Transportation Knowledge of  (Multimodal) Large Language Models

The blockchain technology provides integrity and reliability of the
information, thus offering a suitable solution to guarantee trustability in a
multi-stakeholder scenario that involves actors defining business agreements.
The Ride2Rail project investigated the use of the blockchain to record as smart
contracts the agreements between different stakeholders defined in a multimodal
transportation domain. Modelling an ontology to represent the smart contracts
enables the possibility of having a machine-readable and interoperable
representation of the agreements. On one hand, the underlying blockchain
ensures trust in the execution of the contracts, on the other hand, their
ontological representation facilitates the retrieval of information within the
ecosystem. The paper describes the development of the Ride2Rail Ontology for
Agreements to showcase how the concept of an ontological smart contract,
defined in the OASIS ontology, can be applied to a specific domain. The usage
of the designed ontology is discussed by describing the modelling as
ontological smart contracts of business agreements defined in a ride-sharing
scenario.

本论文探讨了在智能合同中使用区块链技术创建本体模型的可能性，以实现跨领域的机器可读和互操作性的协议。研究以一个多利益相关者场景作为案例，旨在解决协议可信度问题。