Natural language generation (NLG) has received increasing attention, which
has highlighted evaluation as a central methodological concern. Since human
evaluations for these systems are costly, automatic metrics have broad appeal
in NLG. Research in language generation often finds situations where it is
appropriate to apply existing metrics or propose new ones. The application of
these metrics are entirely dependent on validation studies - studies that
determine a metric's correlation to human judgment. However, there are many
details and considerations in conducting strong validation studies. This
document is intended for those validating existing metrics or proposing new
ones in the broad context of NLG: we 1) begin with a write-up of best practices
in validation studies, 2) outline how to adopt these practices, 3) conduct
analyses in the WMT'17 metrics shared task\footnote{Our jupyter notebook
containing the analyses is available at https://github.com}, and 4)
highlight promising approaches to NLG metrics 5) conclude with our opinions on
the future of this area.

本文主要探讨自然语言生成领域中测评方法中的自动指标的应用和验证，提出了验证研究的最佳实践，并在 WMT'17 度量共享任务中进行了分析，同时也突出了未来的发展方向。