Finetuned large language models (such as ChatGPT and Qwen-chat) can generate
Chinese classical poetry following human's instructions. LLMs perform well in
content, but are usually lacking in format, with occasionally excess or
insufficient number of characters in each line. Since most SOTA LLMs are
token-based, we assume that the format inaccuracy is due to the difficulty of
the "token planning" task, which means that the LLM need to know exactly how
much characters are contained in each token and do length-control planning
based on that knowledge. In this paper, we first confirm our assumption by
showing that existing token-based large language models has limited knowledge
on token-character relationship. We use a spelling bee probing procedure, and
find that Qwen-chat failed in nearly 15% Chinese spelling test. We then show
that a token-based model can be easily tailored into a token-free model (in
terms of Chinese), which can largely solve the format accuracy problem. Our
tailoring procedure removes long-token from vocabulary and keeps only
character-level or byte-level tokens. As part of our contribution, we release
the finetuned token-free model (which is based on Qwen-chat-7B), which can
generate chinese classical poetry following complex instructions like LLMs
(such as story paraphrasing), and also perform well in format. On the test set,
our token-free model achives an format accuracy of 0.96, compared to 0.84 for
token-based counterparts and 0.38 for GPT-4.

本研究通过验证现行基于标记的大型语言模型在标记 - 字符关系方面的知识限制，并提出一种基于字符级或字节级标记的无标记模型，该模型在格式准确性方面表现优异，为解决生成中文古典诗歌的格式问题提供了一个可行的解决方案。