Image-guided object assembly represents a burgeoning research topic in
computer vision. This paper introduces a novel task: translating multi-view
images of a structural 3D model (for example, one constructed with building
blocks drawn from a 3D-object library) into a detailed sequence of assembly
instructions executable by a robotic arm. Fed with multi-view images of the
target 3D model for replication, the model designed for this task must address
several sub-tasks, including recognizing individual components used in
constructing the 3D model, estimating the geometric pose of each component, and
deducing a feasible assembly order adhering to physical rules. Establishing
accurate 2D-3D correspondence between multi-view images and 3D objects is
technically challenging. To tackle this, we propose an end-to-end model known
as the Neural Assembler. This model learns an object graph where each vertex
represents recognized components from the images, and the edges specify the
topology of the 3D model, enabling the derivation of an assembly plan. We
establish benchmarks for this task and conduct comprehensive empirical
evaluations of Neural Assembler and alternative solutions. Our experiments
clearly demonstrate the superiority of Neural Assembler.

一篇关于图像引导物体组装的研究论文介绍了一项新任务：将结构 3D 模型的多视角图像（例如，从 3D 对象库中选择的积木构建而成的模型）翻译成可由机器人臂执行的详细组装指令序列。该论文提出了一种名为神经组装者的端到端模型，通过学习图形对象图来识别图像中的组件，并通过边指定 3D 模型的拓扑结构，从而得出组装计划，并进行了全面的实证评估，证明了神经组装者的优越性。

神经装配器：从多视图图像中学习生成细粒度的机器人组装指令

Neural Assembler: Learning to Generate Fine-Grained Robotic Assembly  Instructions from Multi-View Images

The assembly instruction is a mandatory component of Lego-like brick sets.The
conventional production of assembly instructions requires a considerable amount
of manual fine-tuning, which is intractable for casual users and customized
brick sets.Moreover, the traditional paper-based instructions lack
expressiveness and interactivity.To tackle the two problems above, we present
BrickPal, an augmented reality-based system, which visualizes assembly
instructions in an augmented reality head-mounted display. It utilizes Natural
Language Processing (NLP) techniques to generate plausible assembly sequences,
and provide real-time guidance in the AR headset.Our user study demonstrates
BrickPal's effectiveness at assisting users in brick assembly compared to
traditional assembly methods. Additionally, the NLP algorithm-generated
assembly sequences achieve the same usability with manually adapted sequences.

利用增强现实技术的 BrickPal 系统，结合自然语言处理技术生成装配指南，通过 AR 头显提供实时指导，有效地帮助用户进行积木组装，NLP 算法生成的装配序列与人工调整的序列有相同的可用性。

BrickPal：基于增强现实的积木模型装配指南

BrickPal: Augmented Reality-based Assembly Instructions for Brick Models

We take the first step to address the task of automatically generating
shellcodes, i.e., small pieces of code used as a payload in the exploitation of
a software vulnerability, starting from natural language comments. We assemble
and release a novel dataset (Shellcode_IA32), consisting of challenging but
common assembly instructions with their natural language descriptions. We
experiment with standard methods in neural machine translation (NMT) to
establish baseline performance levels on this task.

本研究的主要目标是利用自然语言注释自动生成用于软件漏洞利用的负载代码，我们收集了一个挑战性的数据集并尝试使用神经机器翻译方法建立基准性能水平。