Generative AI such as those with large language models have created
opportunities for innovative assessment design practices. Due to recent
technological developments, there is a need to know the limits and capabilities
of generative AI in terms of simulating cognitive skills. Assessing student
critical thinking skills has been a feature of assessment for time immemorial,
but the demands of digital assessment create unique challenges for equity,
academic integrity and assessment authorship. Educators need a framework for
determining their assessments vulnerability to generative AI to inform
assessment design practices. This paper presents a framework that explores the
capabilities of the LLM ChatGPT4 application, which is the current industry
benchmark. This paper presents the Mapping of questions, AI vulnerability
testing, Grading, Evaluation (MAGE) framework to methodically critique their
assessments within their own disciplinary contexts. This critique will provide
specific and targeted indications of their questions vulnerabilities in terms
of the critical thinking skills. This can go on to form the basis of assessment
design for their tasks.

通过 MAGE 框架，本研究旨在探索 LLM ChatGPT4 应用的能力，为教育工作者提供一个方法，以在其学科背景中系统地评估其评估的脆弱性，并提供特定和针对性的问题脆弱性指示，以评估关键思维技能的基础。