Locating objects referred to in natural language poses a significant
challenge for autonomous agents. Existing CLIP-based open-vocabulary methods
successfully perform 3D object retrieval with simple (bare) queries but cannot
cope with ambiguous descriptions that demand an understanding of object
relations. To tackle this problem, we propose a modular approach called BBQ
(Beyond Bare Queries), which constructs 3D scene spatial graph representation
with metric edges and utilizes a large language model as a human-to-agent
interface through our deductive scene reasoning algorithm. BBQ employs robust
DINO-powered associations to form 3D objects, an advanced raycasting algorithm
to project them to 2D, and a vision-language model to describe them as graph
nodes. On Replica and ScanNet datasets, we show that the designed method
accurately constructs 3D object-centric maps. We have demonstrated that their
quality takes a leading place for open-vocabulary 3D semantic segmentation
against other zero-shot methods. Also, we show that leveraging spatial
relations is especially effective for scenes containing multiple entities of
the same semantic class. On Sr3D and Nr3D benchmarks, our deductive approach
demonstrates a significant improvement, enabling retrieving objects by complex
queries compared to other state-of-the-art methods. Considering our design
solutions, we achieved a processing speed approximately x3 times faster than
the closest analog. This promising performance enables our approach for usage
in applied intelligent robotics projects. We make the code publicly available
at linukc.github.io/bbq/.

通过提出模块化方法 BBQ（Beyond Bare Queries），结合构建 3D 场景空间图表示、套用大型语言模型以及深度推理算法，本研究成功实现了生成准确的 3D 物体中心地图、应用于开放词汇 3D 语义分割、并具备在复杂查询中通过使用空间关系检索物体的能力，并且其处理速度比最近的类似方法快大约三倍。

超越裸露询问：使用 3D 场景图进行开放词汇物体检索

Beyond Bare Queries: Open-Vocabulary Object Retrieval with 3D Scene  Graph

Anomaly detection in chest X-rays is a critical task. Most methods mainly
model the distribution of normal images, and then regard significant deviation
from normal distribution as anomaly. Recently, CLIP-based methods, pre-trained
on a large number of medical images, have shown impressive performance on
zero/few-shot downstream tasks. In this paper, we aim to explore the potential
of CLIP-based methods for anomaly detection in chest X-rays. Considering the
discrepancy between the CLIP pre-training data and the task-specific data, we
propose a position-guided prompt learning method. Specifically, inspired by the
fact that experts diagnose chest X-rays by carefully examining distinct lung
regions, we propose learnable position-guided text and image prompts to adapt
the task data to the frozen pre-trained CLIP-based model. To enhance the
model's discriminative capability, we propose a novel structure-preserving
anomaly synthesis method within chest x-rays during the training process.
Extensive experiments on three datasets demonstrate that our proposed method
outperforms some state-of-the-art methods. The code of our implementation is
available at this https URL

该研究论文探讨了基于 CLIP 方法在胸部 X 射线异常检测中的潜力，并提出了一种基于位置引导的提示学习方法和一种新颖的结构保持异常合成方法，实验证明这两种方法优于一些现有方法。

胸片异常检测的位置指导式提示学习

Position-Guided Prompt Learning for Anomaly Detection in Chest X-Rays

Recent progress in the text-driven 3D stylization of a single object has been
considerably promoted by CLIP-based methods. However, the stylization of
multi-object 3D scenes is still impeded in that the image-text pairs used for
pre-training CLIP mostly consist of an object. Meanwhile, the local details of
multiple objects may be susceptible to omission due to the existing supervision
manner primarily relying on coarse-grained contrast of image-text pairs. To
overcome these challenges, we present a novel framework, dubbed TeMO, to parse
multi-object 3D scenes and edit their styles under the contrast supervision at
multiple levels. We first propose a Decoupled Graph Attention (DGA) module to
distinguishably reinforce the features of 3D surface points. Particularly, a
cross-modal graph is constructed to align the object points accurately and noun
phrases decoupled from the 3D mesh and textual description. Then, we develop a
Cross-Grained Contrast (CGC) supervision system, where a fine-grained loss
between the words in the textual description and the randomly rendered images
are constructed to complement the coarse-grained loss. Extensive experiments
show that our method can synthesize high-quality stylized content and
outperform the existing methods over a wide range of multi-object 3D meshes.
Our code and results will be made publicly available

通过 TeMO 框架，解决了多对象 3D 场景的样式化问题，并利用交叉粗粒度对比监督系统构建细粒度损失，提高了多对象 3D 网格样式化的质量。

TeMO: 文本驱动的多物体网格三维风格化

TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes

For text-to-video retrieval (T2VR), which aims to retrieve unlabeled videos
by ad-hoc textual queries, CLIP-based methods are dominating. Compared to
CLIP4Clip which is efficient and compact, the state-of-the-art models tend to
compute video-text similarity by fine-grained cross-modal feature interaction
and matching, putting their scalability for large-scale T2VR into doubt. For
efficient T2VR, we propose TeachCLIP with multi-grained teaching to let a
CLIP4Clip based student network learn from more advanced yet computationally
heavy models such as X-CLIP, TS2-Net and X-Pool . To improve the student's
learning capability, we add an Attentional frame-Feature Aggregation (AFA)
block, which by design adds no extra storage/computation overhead at the
retrieval stage. While attentive weights produced by AFA are commonly used for
combining frame-level features, we propose a novel use of the weights to let
them imitate frame-text relevance estimated by the teacher network. As such,
AFA provides a fine-grained learning (teaching) channel for the student
(teacher). Extensive experiments on multiple public datasets justify the
viability of the proposed method.

通过多层次教学，利用具有更高性能但计算负荷较大的模型，如 X-CLIP、TS2-Net 和 X-Pool，以及引入注意力框架特征聚合（AFA）块来提高学生网络的学习能力，以实现高效的文本到视频检索。实验结果表明所提出的方法是可行的。