In medical and industrial domains, providing guidance for assembly processes
is critical to ensure efficiency and safety. Errors in assembly can lead to
significant consequences such as extended surgery times, and prolonged
manufacturing or maintenance times in industry. Assembly scenarios can benefit
from in-situ AR visualization to provide guidance, reduce assembly times and
minimize errors. To enable in-situ visualization 6D pose estimation can be
leveraged. Existing 6D pose estimation techniques primarily focus on individual
objects and static captures. However, assembly scenarios have various dynamics
including occlusion during assembly and dynamics in the assembly objects
appearance. Existing work, combining object detection/6D pose estimation and
assembly state detection focuses either on pure deep learning-based approaches,
or limit the assembly state detection to building blocks. To address the
challenges of 6D pose estimation in combination with assembly state detection,
our approach ASDF builds upon the strengths of YOLOv8, a real-time capable
object detection framework. We extend this framework, refine the object pose
and fuse pose knowledge with network-detected pose information. Utilizing our
late fusion in our Pose2State module results in refined 6D pose estimation and
assembly state detection. By combining both pose and state information, our
Pose2State module predicts the final assembly state with precision. Our
evaluation on our ASDF dataset shows that our Pose2State module leads to an
improved assembly state detection and that the improvement of the assembly
state further leads to a more robust 6D pose estimation. Moreover, on the GBOT
dataset, we outperform the pure deep learning-based network, and even
outperform the hybrid and pure tracking-based approaches.

提供医疗和工业领域的装配过程指导对于确保效率和安全至关重要，装配场景可以通过原位增强现实可视化提供指导，减少装配时间和错误，并结合对象检测、装配状态检测、对象姿态估计等技术，实现精确的最终装配状态预测。