Engineering Design is undergoing a transformative shift with the advent of
AI, marking a new era in how we approach product, system, and service planning.
Large language models have demonstrated impressive capabilities in enabling
this shift. Yet, with text as their only input modality, they cannot leverage
the large body of visual artifacts that engineers have used for centuries and
are accustomed to. This gap is addressed with the release of multimodal vision
language models, such as GPT-4V, enabling AI to impact many more types of
tasks. In light of these advancements, this paper presents a comprehensive
evaluation of GPT-4V, a vision language model, across a wide spectrum of
engineering design tasks, categorized into four main areas: Conceptual Design,
System-Level and Detailed Design, Manufacturing and Inspection, and Engineering
Education Tasks. Our study assesses GPT-4V's capabilities in design tasks such
as sketch similarity analysis, concept selection using Pugh Charts, material
selection, engineering drawing analysis, CAD generation, topology optimization,
design for additive and subtractive manufacturing, spatial reasoning
challenges, and textbook problems. Through this structured evaluation, we not
only explore GPT-4V's proficiency in handling complex design and manufacturing
challenges but also identify its limitations in complex engineering design
applications. Our research establishes a foundation for future assessments of
vision language models, emphasizing their immense potential for innovating and
enhancing the engineering design and manufacturing landscape. It also
contributes a set of benchmark testing datasets, with more than 1000 queries,
for ongoing advancements and applications in this field.

利用 GPT-4V 模型进行了一项广泛评估，涵盖概念设计、系统级和详细设计、制造和检验以及工程教育等四个主要领域的工程设计任务，在分析中发现了该模型在处理复杂设计和制造挑战方面的能力，并确定了其在复杂工程设计应用中的局限性。