We introduce a novel writing method called Probing Chain of Thought (ProCoT),
which prevents students from cheating using a Large Language Model (LLM), such
as ChatGPT, while enhancing their active learning through such models. LLMs
have disrupted education and many other feilds. For fear of students cheating,
many educationists have resorted to banning their use, as their outputs can be
human-like and hard to detect in some cases. These LLMs are also known for
hallucinations (i.e. fake facts). We conduct studies with ProCoT in two
different courses with a combined total of about 66 students. The students in
each course were asked to prompt an LLM of their choice with one question from
a set of four and required to affirm or refute statements in the LLM output by
using peer reviewed references. The results show two things: (1) ProCoT
stimulates creative/critical thinking and writing of students through
engagement with LLMs when we compare the LLM solely output to ProCoT output and
(2) ProCoT can prevent cheating because of clear limitations in existing LLMs
when we compare students ProCoT output to LLM ProCoT output. We also discover
that most students prefer to give answers in fewer words than LLMs, which are
typically verbose. The average word counts for students, ChatGPT (v3.5) and
Phind (v8) are 208, 391 and 383, respectively.

介绍了一种名为 Probing Chain of Thought（ProCoT）的新型写作方法，可在防止学生利用大型语言模型（LLM）作弊的同时，通过这些模型增强他们的主动学习。研究发现，ProCoT 可激发学生的创造性 / 批判性思维和写作能力，并可防止作弊，同时揭示了现有 LLM 的局限性。