Generative AI such as those with large language models have created
opportunities for innovative assessment design practices. Due to recent
technological developments, there is a need to know the limits and capabilities
of generative AI in terms of simulating cognitive skills. Assessing student
critical thinking skills has been a feature of assessment for time immemorial,
but the demands of digital assessment create unique challenges for equity,
academic integrity and assessment authorship. Educators need a framework for
determining their assessments vulnerability to generative AI to inform
assessment design practices. This paper presents a framework that explores the
capabilities of the LLM ChatGPT4 application, which is the current industry
benchmark. This paper presents the Mapping of questions, AI vulnerability
testing, Grading, Evaluation (MAGE) framework to methodically critique their
assessments within their own disciplinary contexts. This critique will provide
specific and targeted indications of their questions vulnerabilities in terms
of the critical thinking skills. This can go on to form the basis of assessment
design for their tasks.

通过 MAGE 框架，本研究旨在探索 LLM ChatGPT4 应用的能力，为教育工作者提供一个方法，以在其学科背景中系统地评估其评估的脆弱性，并提供特定和针对性的问题脆弱性指示，以评估关键思维技能的基础。

一种评估生成型人工智能思考质量的框架

How critically can an AI think? A framework for evaluating the quality  of thinking of generative artificial intelligence

Dialogue-based Intelligent Tutoring Systems (ITSs) have significantly
advanced adaptive and personalized learning by automating sophisticated human
tutoring strategies within interactive dialogues. However, replicating the
nuanced patterns of expert human communication remains a challenge in Natural
Language Processing (NLP). Recent advancements in NLP, particularly Large
Language Models (LLMs) such as OpenAI's GPT-4, offer promising solutions by
providing human-like and context-aware responses based on extensive pre-trained
knowledge. Motivated by the effectiveness of LLMs in various educational tasks
(e.g., content creation and summarization, problem-solving, and automated
feedback provision), our study introduces the Socratic Playground for Learning
(SPL), a dialogue-based ITS powered by the GPT-4 model, which employs the
Socratic teaching method to foster critical thinking among learners. Through
extensive prompt engineering, SPL can generate specific learning scenarios and
facilitates efficient multi-turn tutoring dialogues. The SPL system aims to
enhance personalized and adaptive learning experiences tailored to individual
needs, specifically focusing on improving critical thinking skills. Our pilot
experimental results from essay writing tasks demonstrate SPL has the potential
to improve tutoring interactions and further enhance dialogue-based ITS
functionalities. Our study, exemplified by SPL, demonstrates how LLMs enhance
dialogue-based ITSs and expand the accessibility and efficacy of educational
technologies.

通过基于 GPT-4 模型的苏格拉底学习法的对话式智能辅导系统（SPL），我们的研究展示了大型语言模型如何增强基于对话的智能辅导系统，并扩展了教育技术的可访问性和效力。

SPL: 由大型语言模型驱动的苏格拉底学习场

SPL: A Socratic Playground for Learning Powered by Large Language Mode

The use of argumentation in education has been shown to improve critical
thinking skills for end-users such as students, and computational models for
argumentation have been developed to assist in this process. Although these
models are useful for evaluating the quality of an argument, they oftentimes
cannot explain why a particular argument is considered poor or not, which makes
it difficult to provide constructive feedback to users to strengthen their
critical thinking skills. In this survey, we aim to explore the different
dimensions of feedback (Richness, Visualization, Interactivity, and
Personalization) provided by the current computational models for
argumentation, and the possibility of enhancing the power of explanations of
such models, ultimately helping learners improve their critical thinking
skills.

通过调查现有的计算模型以及反驳理论中不同维度的反馈（丰富性、可视化、交互性和个人化），探讨提供解释能力的可能性，进而帮助学习者提高批判性思维能力。