There is an increasing interest in leveraging Large Language Models (LLMs)
for managing structured data and enhancing data science processes. Despite the
potential benefits, this integration poses significant questions regarding
their reliability and decision-making methodologies. It highlights the
importance of various factors in the model selection process, including the
nature of the data, problem type, performance metrics, computational resources,
interpretability vs accuracy, assumptions about data, and ethical
considerations. Our objective is to elucidate and express the factors and
assumptions guiding GPT-4's model selection recommendations. We employ a
variability model to depict these factors and use toy datasets to evaluate both
the model and the implementation of the identified heuristics. By contrasting
these outcomes with heuristics from other platforms, our aim is to determine
the effectiveness and distinctiveness of GPT-4's methodology. This research is
committed to advancing our comprehension of AI decision-making processes,
especially in the realm of model selection within data science. Our efforts are
directed towards creating AI systems that are more transparent and
comprehensible, contributing to a more responsible and efficient practice in
data science.

利用大型语言模型 (LLMs) 管理结构化数据并增强数据科学流程的兴趣日益增长。尽管具有潜在的好处，但其整合引发了对可靠性和决策方法的重要问题，强调了模型选择过程中包括数据性质、问题类型、性能指标、计算资源、可解释性与准确性、对数据的假设以及伦理考虑等多种因素的重要性。我们的目标是阐明和表达 GPT-4 模型选择推荐背后的因素和假设。我们采用变异性模型描述这些因素，并使用玩具数据集评估模型和已确定的启发式方法的实施。通过将这些结果与其他平台的启发式方法进行对比，我们的目的是确定 GPT-4 方法的有效性和独特性。本研究致力于推进我们对人工智能决策过程的理解，特别是在数据科学中的模型选择领域。我们的努力旨在创建更加透明和可理解的人工智能系统，为数据科学实践贡献更负责任和高效的方法。