Hallucinations in vision-language models pose a significant challenge to
their reliability, particularly in the generation of long captions. Current
methods fall short of accurately identifying and mitigating these
hallucinations. To address this issue, we introduce ESREAL, a novel
unsupervised learning framework designed to suppress the generation of
hallucinations through accurate localization and penalization of hallucinated
tokens. Initially, ESREAL creates a reconstructed image based on the generated
caption and aligns its corresponding regions with those of the original image.
This semantic reconstruction aids in identifying both the presence and type of
token-level hallucinations within the generated caption. Subsequently, ESREAL
computes token-level hallucination scores by assessing the semantic similarity
of aligned regions based on the type of hallucination. Finally, ESREAL employs
a proximal policy optimization algorithm, where it selectively penalizes
hallucinated tokens according to their token-level hallucination scores. Our
framework notably reduces hallucinations in LLaVA, InstructBLIP, and mPLUG-Owl2
by 32.81%, 27.08%, and 7.46% on the CHAIR metric. This improvement is achieved
solely through signals derived from the image itself, without the need for any
image-text pairs.

通过准确定位和惩罚产生错觉的标记，ESREAL 提出了一种新颖的无监督学习框架，以抑制视觉 - 语言模型在生成长字幕时的幻觉，通过分析图像本身的信号实现减少错觉的目标。

利用语义重建缓解视觉语言模型中的幻像

Exploiting Semantic Reconstruction to Mitigate Hallucinations in  Vision-Language Models

This paper explores the problem of reconstructing temporally consistent
surfaces from a 3D point cloud sequence without correspondence. To address this
challenging task, we propose DynoSurf, an unsupervised learning framework
integrating a template surface representation with a learnable deformation
field. Specifically, we design a coarse-to-fine strategy for learning the
template surface based on the deformable tetrahedron representation.
Furthermore, we propose a learnable deformation representation based on the
learnable control points and blending weights, which can deform the template
surface non-rigidly while maintaining the consistency of the local shape.
Experimental results demonstrate the significant superiority of DynoSurf over
current state-of-the-art approaches, showcasing its potential as a powerful
tool for dynamic mesh reconstruction. The code is publicly available at
this https URL

本文探讨了在没有对应关系的情况下，如何从 3D 点云序列中重构出时间连贯的表面。为了解决这个具有挑战性的任务，我们提出了 DynoSurf，这是一个融合可学习变形场和模板表面表示的无监督学习框架。具体而言，我们设计了一种基于可变形四面体表示的粗到细的学习模板表面的策略。此外，我们提出了一种基于可学习控制点和混合权重的可变形表示，可以在保持局部形状一致性的同时非刚性地变形模板表面。实验结果表明，DynoSurf 相对于当前最先进的方法具有显著的优势，展示了其作为动态网格重构强大工具的潜力。该代码可以公开获取于此链接。

DynoSurf: 基于神经变形的时间一致性动态表面重建

DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface  Reconstruction

Hyperspectral images (HSI) have become popular for analysing remotely sensed
images in multiple domain like agriculture, medical. However, existing models
struggle with complex relationships and characteristics of spectral-spatial
data due to the multi-band nature and data redundancy of hyperspectral data. To
address this limitation, we propose a new network called DiffSpectralNet, which
combines diffusion and transformer techniques. Our approach involves a two-step
process. First, we use an unsupervised learning framework based on the
diffusion model to extract both high-level and low-level spectral-spatial
features. The diffusion method is capable of extracting diverse and meaningful
spectral-spatial features, leading to improvement in HSI classification. Then,
we employ a pretrained denoising U-Net to extract intermediate hierarchical
features for classification. Finally, we use a supervised transformer-based
classifier to perform the HSI classification. Through comprehensive experiments
on HSI datasets, we evaluate the classification performance of DiffSpectralNet.
The results demonstrate that our framework significantly outperforms existing
approaches, achieving state-of-the-art performance.

提出了一种名为 DiffSpectralNet 的新型网络，通过结合扩散和变换器技术，利用无监督学习和预训练去噪 U-Net 提取高层和低层光谱空间特征，并使用监督变换器分类器进行高光谱图像分类，实现了超越现有方法的创新，达到了最先进的性能。