Contacting customer service via chat is a common practice. Because employing
customer service agents is expensive, many companies are turning to NLP that
assists human agents by auto-generating responses that can be used directly or
with modifications. Large Language Models (LLMs) are a natural fit for this use
case; however, their efficacy must be balanced with the cost of training and
serving them. This paper assesses the practical cost and impact of LLMs for the
enterprise as a function of the usefulness of the responses that they generate.
We present a cost framework for evaluating an NLP model's utility for this use
case and apply it to a single brand as a case study in the context of an
existing agent assistance product. We compare three strategies for specializing
an LLM - prompt engineering, fine-tuning, and knowledge distillation - using
feedback from the brand's customer service agents. We find that the usability
of a model's responses can make up for a large difference in inference cost for
our case study brand, and we extrapolate our findings to the broader enterprise
space.

使用大型语言模型（LLMs）辅助人类客服代理商自动生成现成或有改动的响应已成为许多公司降低雇用客服代理商成本的方式。本文通过使用成本框架评估一个 NLP 模型的实用性，将其应用于特定品牌的案例研究并比较三种 LLM 的专业化策略，发现模型的响应可用性可以补偿推理成本的巨大差异，并将研究结果推广到更广泛的企业领域。