While ChatGPT is a well-known artificial intelligence chatbot being used to
answer human's questions, one may want to discover its potential in advancing
software testing. We examine the capability of ChatGPT in advancing the
intelligence of software testing through a case study on metamorphic testing
(MT), a state-of-the-art software testing technique. We ask ChatGPT to generate
candidates of metamorphic relations (MRs), which are basically necessary
properties of the object program and which traditionally require human
intelligence to identify. These MR candidates are then evaluated in terms of
correctness by domain experts. We show that ChatGPT can be used to generate new
correct MRs to test several software systems. Having said that, the majority of
MR candidates are either defined vaguely or incorrect, especially for systems
that have never been tested with MT. ChatGPT can be used to advance software
testing intelligence by proposing MR candidates that can be later adopted for
implementing tests; but human intelligence should still inevitably be involved
to justify and rectify their correctness.

ChatGPT 能用来推动软件测试智能化，尤其是在元胞变换测试方面，它可以生成测试新的元胞关系候选项，但需要人工智能的参与来确保其正确性。

ChatGPT 能推动软件测试智能化发展吗？基于变异测试的经验报告

Can ChatGPT advance software testing intelligence? An experience report  on metamorphic testing

Quantization is one of the most applied Deep Neural Network (DNN) compression
strategies, when deploying a trained DNN model on an embedded system or a cell
phone. This is owing to its simplicity and adaptability to a wide range of
applications and circumstances, as opposed to specific Artificial Intelligence
(AI) accelerators and compilers that are often designed only for certain
specific hardware (e.g., Google Coral Edge TPU). With the growing demand for
quantization, ensuring the reliability of this strategy is becoming a critical
challenge. Traditional testing methods, which gather more and more genuine data
for better assessment, are often not practical because of the large size of the
input space and the high similarity between the original DNN and its quantized
counterpart. As a result, advanced assessment strategies have become of
paramount importance. In this paper, we present DiverGet, a search-based
testing framework for quantization assessment. DiverGet defines a space of
metamorphic relations that simulate naturally-occurring distortions on the
inputs. Then, it optimally explores these relations to reveal the disagreements
among DNNs of different arithmetic precision. We evaluate the performance of
DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing
images. We chose the remote sensing DNNs as they're being increasingly deployed
at the edge (e.g., high-lift drones) in critical domains like climate change
research and astronomy. Our results show that DiverGet successfully challenges
the robustness of established quantization techniques against
naturally-occurring shifted data, and outperforms its most recent concurrent,
DiffChaser, with a success rate that is (on average) four times higher.

本文提出基于搜索的 DiverGet 测试框架，通过定义一组模拟数据失真的变形关系空间，优化地探索这些关系来揭示不同算术精度的 DNN 之间的差异，对 Hyperspectral Remote Sensing 图像的最新 DNN 进行评估，结果表明 DiverGet 成功挑战了已有的量化技术对自然发生的平移数据的鲁棒性，优于其最近的竞争对手 DiffChaser。