The Mixture-of-Experts (MoE) has gained increasing attention in the study of
Large Vision-Language Models (LVLMs). It uses a sparse model to replace the
dense model, achieving comparable performance while activating fewer parameters
during inference, thus significantly reducing the inference cost. Existing MoE
methods in LVLMs encourage different experts to handle different tokens, and
thus they employ a router to predict the routing for each token. However, the
predictions are based solely on sample features and do not truly reveal the
optimization direction of tokens. This can lead to severe optimization
conflicts between different tokens within an expert. To address this problem,
this paper proposes a novel method based on token-level gradient analysis.
Specifically, we first use token-level gradients to identify conflicting tokens
in experts. Then, we add a specialized loss tailored to eliminate conflicts
among tokens within each expert. Our method can serve as a plug-in for diverse
Large Vision-Language Models, and extensive experimental results demonstrate
the effectiveness of our method. The code will be publicly available at
this https URL

使用基于令牌级梯度分析的新方法，该文章旨在解决大型视觉语言模型中令牌优化冲突的问题，并通过消除专家内部的令牌冲突来提高性能。

大规模视觉语言模型中的混合专家中的令牌梯度冲突解决

Solving Token Gradient Conflict in Mixture-of-Experts for Large  Vision-Language Model

We delve into pseudo-labeling for semi-supervised monocular 3D object
detection (SSM3OD) and discover two primary issues: a misalignment between the
prediction quality of 3D and 2D attributes and the tendency of depth
supervision derived from pseudo-labels to be noisy, leading to significant
optimization conflicts with other reliable forms of supervision. We introduce a
novel decoupled pseudo-labeling (DPL) approach for SSM3OD. Our approach
features a Decoupled Pseudo-label Generation (DPG) module, designed to
efficiently generate pseudo-labels by separately processing 2D and 3D
attributes. This module incorporates a unique homography-based method for
identifying dependable pseudo-labels in BEV space, specifically for 3D
attributes. Additionally, we present a DepthGradient Projection (DGP) module to
mitigate optimization conflicts caused by noisy depth supervision of
pseudo-labels, effectively decoupling the depth gradient and removing
conflicting gradients. This dual decoupling strategy-at both the pseudo-label
generation and gradient levels-significantly improves the utilization of
pseudo-labels in SSM3OD. Our comprehensive experiments on the KITTI benchmark
demonstrate the superiority of our method over existing approaches.

针对半监督单目三维物体检测，我们研究了伪标记方法，并发现了两个主要问题：三维和二维属性预测质量之间的不对齐以及伪标签导致的深度监督存在噪声，从而与其他可靠的监督形式产生显著的优化冲突。我们提出了一种新颖的解耦伪标记方法，该方法包括一个用于分别处理二维和三维属性的解耦伪标签生成模块和一个用于减轻由伪标签的深度监督引起的优化冲突的深度梯度投影模块。在 KITTI 基准测试上的综合实验证明了我们方法的优越性。