Deep learning models, particularly those based on transformers, often employ
numerous stacked structures, which possess identical architectures and perform
similar functions. While effective, this stacking paradigm leads to a
substantial increase in the number of parameters, posing challenges for
practical applications. In today's landscape of increasingly large models,
stacking depth can even reach dozens, further exacerbating this issue. To
mitigate this problem, we introduce LORS (LOw-rank Residual Structure). LORS
allows stacked modules to share the majority of parameters, requiring a much
smaller number of unique ones per module to match or even surpass the
performance of using entirely distinct ones, thereby significantly reducing
parameter usage. We validate our method by applying it to the stacked decoders
of a query-based object detector, and conduct extensive experiments on the
widely used MS COCO dataset. Experimental results demonstrate the effectiveness
of our method, as even with a 70\% reduction in the parameters of the decoder,
our method still enables the model to achieve comparable or

通过引入 LOw-rank Residual Structure (LORS) 方法，该研究论文探讨了如何降低深度学习模型的参数数量并提高性能，以在 query-based 目标检测器的堆叠解码器中实现参数节省，实验结果表明该方法能够使模型在减少 70% 解码器参数的情况下，仍能达到相当或更好的性能。