Semiconductor imaging and analysis are critical yet understudied in deep learning, limiting our ability for precise control and optimization in semiconductor manufacturing. We introduce a small-scale multimodal framework for analyzing semiconductor electron microscopy images (MAEMI) through vision-language instruction tuning. We generate a customized instruction-following dataset using large multimodal models on microscopic image analysis. We perform knowledge transfer from larger to smaller models through knowledge distillation, resulting in improved accuracy of smaller models on visual question answering (VQA) tasks. This approach eliminates the need for expensive, human expert-annotated datasets for microscopic image analysis tasks. Enterprises can further finetune MAEMI on their intellectual data, enhancing privacy and performance on low-cost consumer hardware. Our experiments show that MAEMI outperforms traditional methods, adapts to data distribution shifts, and supports high-throughput screening.

本研究解决了在半导体成像和分析中，深度学习应用不足导致的精确控制和优化问题。通过视觉-语言指令调优，我们提出了一个小型多模态框架（MAEMI）以改进半导体电子显微镜图像分析，显著提高了小型模型在视觉问答任务中的准确性。实验结果表明，MAEMI超越了传统方法，能够适应数据分布变化，并支持高通量筛选，具有良好的企业应用潜力。

电子显微镜分析的基础模型：指令调优小型语言与视觉助手的企业应用