Recent advancements in image segmentation have focused on enhancing the
efficiency of the models to meet the demands of real-time applications,
especially on edge devices. However, existing research has primarily
concentrated on single-task settings, especially on semantic segmentation,
leading to redundant efforts and specialized architectures for different tasks.
To address this limitation, we propose a novel architecture for efficient
multi-task image segmentation, capable of handling various segmentation tasks
without sacrificing efficiency or accuracy. We introduce BiSeNetFormer, that
leverages the efficiency of two-stream semantic segmentation architectures and
it extends them into a mask classification framework. Our approach maintains
the efficient spatial and context paths to capture detailed and semantic
information, respectively, while leveraging an efficient transformed-based
segmentation head that computes the binary masks and class probabilities. By
seamlessly supporting multiple tasks, namely semantic and panoptic
segmentation, BiSeNetFormer offers a versatile solution for multi-task
segmentation. We evaluate our approach on popular datasets, Cityscapes and
ADE20K, demonstrating impressive inference speeds while maintaining competitive
accuracy compared to state-of-the-art architectures. Our results indicate that
BiSeNetFormer represents a significant advancement towards fast, efficient, and
multi-task segmentation networks, bridging the gap between model efficiency and
task adaptability.

提出了一种针对图像分割的高效多任务架构 BiSeNetFormer，能够在保持高效和准确性的同时处理多种分割任务，其结果表明 BiSeNetFormer 在模型效率和任务适应性之间构建了桥梁，具有快速、高效和多任务分割网络的显著进展。

BiSeNet 的复仇：高效的多任务图像分割

The revenge of BiSeNet: Efficient Multi-Task Image Segmentation

Catastrophic forgetting emerges as a critical challenge when fine-tuning
multi-modal large language models (MLLMs), where improving performance on
unseen tasks often leads to a significant performance drop on the original
tasks. This paper presents a comprehensive analysis of catastrophic forgetting
in MLLMs and introduces a post-training adjustment method called Model Tailor.
Our method primarily preserves the pre-trained parameters while replacing a
small number ($\leq$ 10\%) of fine-tuned parameters, maintaining $\sim$ 99\%
effectiveness on original tasks versus pre-training, and achieving $\sim$ 97\%
on new tasks compared to standard fine-tuning. Specifically, we derive a sparse
mask to identify the "model patch", based on a fusion strategy that integrates
salience and sensitivity analysis. Subsequently, a compensation mechanism is
introduced to "decorate the patch", enhancing the model's performance on both
target and original tasks. Additionally, our method is adaptable to multi-task
scenarios. Through extensive experiments on InstructBLIP and LLaVA-1.5 in both
image captioning and visual question answering tasks, our approach demonstrates
significant task adaptability while preserving inherent pre-trained
capabilities.

该论文介绍了多模态大型语言模型（MLLMs）中灾难性遗忘现象的全面分析，并引入了一种名为 Model Tailor 的后训练调整方法。该方法主要保留了预训练参数，同时替换了少量（≤ 10%）的微调参数，相比于预训练，在原始任务上保持了约 99% 的有效性，并在新任务上相比标准微调获得了约 97% 的有效性。此外，该方法适用于多任务场景，并通过在 InstructBLIP 和 LLaVA-1.5 上进行广泛实验证明了显著的任务适应能力，同时保持了内在的预训练能力。