Most existing one-shot skeleton-based action recognition focuses on raw
low-level information (e.g., joint location), and may suffer from local
information loss and low generalization ability. To alleviate these, we propose
to leverage text description generated from large language models (LLM) that
contain high-level human knowledge, to guide feature learning, in a
global-local-global way. Particularly, during training, we design $2$ prompts
to gain global and local text descriptions of each action from an LLM. We first
utilize the global text description to guide the skeleton encoder focus on
informative joints (i.e.,global-to-local). Then we build non-local interaction
between local text and joint features, to form the final global representation
(i.e., local-to-global). To mitigate the asymmetry issue between the training
and inference phases, we further design a dual-branch architecture that allows
the model to perform novel class inference without any text input, also making
the additional inference cost neglectable compared with the base skeleton
encoder. Extensive experiments on three different benchmarks show that CrossGLG
consistently outperforms the existing SOTA methods with large margins, and the
inference cost (model size) is only $2.8$\% than the previous SOTA. CrossGLG
can also serve as a plug-and-play module that can substantially enhance the
performance of different SOTA skeleton encoders with a neglectable cost during
inference. The source code will be released soon.

通过大型语言模型生成的文本描述，全局 - 局部 - 全局方式指导特征学习，设计了双分支架构，提高单次骨架动作识别的准确性和推理效率。

CrossGLG: LLM 以跨层级方式指导一次性基于骨架的 3D 动作识别

CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a  Cross-level Manner

We present a novel method, AerialBooth, for synthesizing the aerial view from
a single input image using its text description. We leverage the pretrained
text-to-2D image stable diffusion model as prior knowledge of the 3D world. The
model is finetuned in two steps to optimize for the text embedding and the UNet
that reconstruct the input image and its inverse perspective mapping
respectively. The inverse perspective mapping creates variance within the
text-image space of the diffusion model, while providing weak guidance for
aerial view synthesis. At inference, we steer the contents of the generated
image towards the input image using novel mutual information guidance that
maximizes the information content between the probability distributions of the
two images. We evaluate our approach on a wide spectrum of real and synthetic
data, including natural scenes, indoor scenes, human action, etc. Through
extensive experiments and ablation studies, we demonstrate the effectiveness of
AerialBooth and also its generalizability to other text-controlled views. We
also show that AerialBooth achieves the best viewpoint-fidelity trade-off
though quantitative evaluation on 7 metrics analyzing viewpoint and fidelity
w.r.t. input image. Code and data is available at
this https URL

通过文本描述合成单个输入图像的空中视图，借助预训练的文本到 2D 图像稳定扩散模型作为 3D 世界的先验知识，经过微调优化文本嵌入和重建输入图像及其逆透视映射的 UNet 模型，在推理中利用互信息指导将生成图像的内容指向输入图像，通过在 7 个度量指标上分析视角和保真度评估显示 AerialBooth 实现了视角和保真度的最佳权衡。

AerialBooth：单幅图像文本控制的航拍视图综合中的互信息导向

AerialBooth: Mutual Information Guidance for Text Controlled Aerial View  Synthesis from a Single Image

Existing neural style transfer methods require reference style images to
transfer texture information of style images to content images. However, in
many practical situations, users may not have reference style images but still
be interested in transferring styles by just imagining them. In order to deal
with such applications, we propose a new framework that enables a style
transfer `without' a style image, but only with a text description of the
desired style. Using the pre-trained text-image embedding model of CLIP, we
demonstrate the modulation of the style of content images only with a single
text condition. Specifically, we propose a patch-wise text-image matching loss
with multiview augmentations for realistic texture transfer. Extensive
experimental results confirmed the successful image style transfer with
realistic textures that reflect semantic query texts.

提出了一种全新的神经风格迁移框架，使用户能够使用文本描述而非参考图像来实现图像的风格迁移，在实验中证实了通过使用多视图扩充的补丁式文本 - 图像匹配 Loss 来实现对内容图像的样式转移，从而实现了具有语义相似性的现实纹理风格转移。