The wide adoption of Large language models (LLMs) makes their dependability a
pressing concern. Detection of errors is the first step to mitigating their
impact on a system and thus, efficient error detection for LLMs is an important
issue. In many settings, the LLM is considered as a black box with no access to
the internal nodes; this prevents the use of many error detection schemes that
need access to the model's internal nodes. An interesting observation is that
the output of LLMs in error-free operation should be valid and normal text.
Therefore, when the text is not valid or differs significantly from normal
text, it is likely that there is an error. Based on this observation we propose
to perform Concurrent Linguistic Error Detection (CLED); this scheme extracts
some linguistic features of the text generated by the LLM and feeds them to a
concurrent classifier that detects errors. Since the proposed error detection
mechanism only relies on the outputs of the model, then it can be used on LLMs
in which there is no access to the internal nodes. The proposed CLED scheme has
been evaluated on the T5 model when used for news summarization and on the
OPUS-MT model when used for translation. In both cases, the same set of
linguistic features has been used for error detection to illustrate the
applicability of the proposed scheme beyond a specific case. The results show
that CLED can detect most of the errors at a low overhead penalty. The use of
the concurrent classifier also enables a trade-off between error detection
effectiveness and its associated overhead, so providing flexibility to a
designer.

基于提取语言模型生成的文本的某些语言特征并将其输入到并发分类器中以检测错误的并发语言错误检测 (CLED) 方案可以在无法访问内部节点的 LLMs 上使用，并且可以在新闻摘要和翻译等不同任务上应用。