Region-level multi-modality methods can translate referred image regions to
human preferred language descriptions. Unfortunately, most of existing methods
using fixed visual inputs remain lacking the resolution adaptability to find
out precise language descriptions. In this study, we propose a dynamic
resolution approach, referred to as DynRefer, to pursue high-accuracy
region-level referring through mimicking the resolution adaptability of human
visual cognition. DynRefer first implements stochastic vision-language
alignment. It aligns desired language descriptions of multi-modality tasks with
images of stochastic resolution, which are constructed by nesting a set of
views around the referred region. DynRefer then implements dynamic
multi-modality referring, which is realized by selecting views based on image
and language priors. This allows the visual information used for referring to
better match human preferences, thereby improving the representational
adaptability of region-level multi-modality models. Extensive experiments show
that DynRefer brings mutual improvement upon tasks including region-level
captioning, open-vocabulary region recognition and attribute detection. Last
but not least, DynRefer achieves new state-of-the-art on multiple region-level
multi-modality tasks using a single model. Code is available at
this https URL

通过动态分辨率方法（DynRefer）来改善区域级多模态任务的高精确度指代，提高多模态模型的表示适应性，并在多个区域级多模态任务上取得新的最先进结果。