This paper aims to quantitatively evaluate the performance of ChatGPT, an
interactive large language model, on inter-sentential relations such as
temporal relations, causal relations, and discourse relations. Given ChatGPT's
promising performance across various tasks, we conduct extensive evaluations on
the whole test sets of 13 datasets, including temporal and causal relations,
PDTB2.0-based and dialogue-based discourse relations, and downstream
applications on discourse understanding. To achieve reliable results, we adopt
three tailored prompt templates for each task, including the zero-shot prompt
template, zero-shot prompt engineering (PE) template, and in-context learning
(ICL) prompt template, to establish the initial baseline scores for all popular
sentence-pair relation classification tasks for the first time. We find that
ChatGPT exhibits strong performance in detecting and reasoning about causal
relations, while it may not be proficient in identifying the temporal order
between two events. It can recognize most discourse relations with existing
explicit discourse connectives, but the implicit discourse relation still
remains a challenging task. Meanwhile, ChatGPT performs poorly in the dialogue
discourse parsing task that requires structural understanding in a dialogue
before being aware of the discourse relation.

该研究针对 ChatGPT 大型语言模型，通过对 13 个数据集的测试集进行综合评估，采用了三种任务定制型 prompt 模板，发现 ChatGPT 对于识别因果关系具有较强的能力，但在识别事件之间的时间顺序方面表现较差，并在需要结构性理解对话的对话语篇分析任务中表现不佳。