In this report, we present our approach to the EPIC-KITCHENS VISOR Hand
Object Segmentation Challenge, which focuses on the estimation of the relation
between the hands and the objects given a single frame as input. The
EPIC-KITCHENS VISOR dataset provides pixel-wise annotations and serves as a
benchmark for hand and active object segmentation in egocentric video. Our
approach combines the baseline method, i.e., Point-based Rendering (PointRend)
and the Segment Anything Model (SAM), aiming to enhance the accuracy of hand
and object segmentation outcomes, while also minimizing instances of missed
detection. We leverage accurate hand segmentation maps obtained from the
baseline method to extract more precise hand and in-contact object segments. We
utilize the class-agnostic segmentation provided by SAM and apply specific
hand-crafted constraints to enhance the results. In cases where the baseline
model misses the detection of hands or objects, we re-train an object detector
on the training set to enhance the detection accuracy. The detected hand and
in-contact object bounding boxes are then used as prompts to extract their
respective segments from the output of SAM. By effectively combining the
strengths of existing methods and applying our refinements, our submission
achieved the 1st place in terms of evaluation criteria in the VISOR HOS
Challenge.

通过结合点渲染和任意物体模型以及应用特定的手工制约条件来增强手部和物体分割结果，在基线模型缺失检测的情况下重新训练物体检测器来提高检测准确性，从而在 VISOR HOS 挑战赛的评估标准中获得第一名。