Punctuation and Segmentation are key to readability in Automatic Speech
Recognition (ASR), often evaluated using F1 scores that require high-quality
human transcripts and do not reflect readability well. Human evaluation is
expensive, time-consuming, and suffers from large inter-observer variability,
especially in conversational speech devoid of strict grammatical structures.
Large pre-trained models capture a notion of grammatical structure. We present
TRScore, a novel readability measure using the GPT model to evaluate different
segmentation and punctuation systems. We validate our approach with human
experts. Additionally, our approach enables quantitative assessment of text
post-processing techniques such as capitalization, inverse text normalization
(ITN), and disfluency on overall readability, which traditional word error rate
(WER) and slot error rate (SER) metrics fail to capture. TRScore is strongly
correlated to traditional F1 and human readability scores, with Pearson's
correlation coefficients of 0.67 and 0.98, respectively. It also eliminates the
need for human transcriptions for model selection.

使用 GPT 模型评估分割和标点系统的新的可读性评估方法：TRScore，可定量评估诸如大写、倒置文本归一化（ITN）和发散性等文本后处理技术对整体可读性的影响，可消除人类转录对模型选择的需求，并且与人类可读性得分和常规 F1 得分存在很强的相关性。