We present TIGERScore, a \textbf{T}rained metric that follows
\textbf{I}nstruction \textbf{G}uidance to perform \textbf{E}xplainable, and
\textbf{R}eference-free evaluation over a wide spectrum of text generation
tasks. Different from other automatic evaluation methods that only provide
arcane scores, TIGERScore is guided by the natural language instruction to
provide error analysis to pinpoint the mistakes in the generated text. Our
metric is based on LLaMA, trained on our meticulously curated
instruction-tuning dataset MetricInstruct which covers 6 text generation tasks
and 23 text generation datasets. The dataset consists of 48K quadruple in the
form of (instruction, input, system output $\rightarrow$ error analysis). We
collected the `system outputs' through diverse channels to cover different
types of errors. To quantitatively assess our metric, we evaluate its
correlation with human ratings on 5 held-in datasets, 2 held-out datasets and
show that TIGERScore can achieve the highest overall Spearman's correlation
with human ratings across these datasets and outperforms other metrics
significantly. As a reference-free metric, its correlation can even surpass the
best existing reference-based metrics. To further qualitatively assess the
rationale generated by our metric, we conduct human evaluation on the generated
explanations and found that the explanations are 70.8\% accurate. Through these
experimental results, we believe TIGERScore demonstrates the possibility of
building universal explainable metrics to evaluate any text generation task.

通过自然语言指示引导的训练度量 TIGERScore 对广泛的文本生成任务进行解释性和无参考评估，该度量基于经过精心策划的指示调优数据集 MetricInstruct 进行 LLaMA 训练，覆盖 6 个文本生成任务和 23 个文本生成数据集，通过与人类评分的相关性评估和定性评估结果表明，TIGERScore 在这些数据集上能够取得最高的斯皮尔曼相关性，显著优于其他度量标准，并且作为无参考度量标准，其相关性甚至可以超过最佳的基于参考的度量标准，通过这些实验结果，我们相信 TIGERScore 展示了建立通用的可解释性度量标准来评估任何文本生成任务的可能性。