By harnessing the capabilities of large language models (LLMs), recent large
multimodal models (LMMs) have shown remarkable versatility in open-world
multimodal understanding. Nevertheless, they are usually parameter-heavy and
computation-intensive, thus hindering their applicability in
resource-constrained scenarios. To this end, several lightweight LMMs have been
proposed successively to maximize the capabilities under constrained scale
(e.g., 3B). Despite the encouraging results achieved by these methods, most of
them only focus on one or two aspects of the design space, and the key design
choices that influence model capability have not yet been thoroughly
investigated. In this paper, we conduct a systematic study for lightweight LMMs
from the aspects of model architecture, training strategy, and training data.
Based on our findings, we obtain Imp -- a family of highly capable LMMs at the
2B-4B scales. Notably, our Imp-3B model steadily outperforms all the existing
lightweight LMMs of similar size, and even surpasses the state-of-the-art LMMs
at the 13B scale. With low-bit quantization and resolution reduction
techniques, our Imp model can be deployed on a Qualcomm Snapdragon 8Gen3 mobile
chip with a high inference speed of about 13 tokens/s.

通过对轻量级大语言模型的系统研究，提出了 Imp 家族高性能的轻量级多模型，并成功在高速移动芯片上进行了部署。

移动设备上高性能的大规模多模态模型

Imp: Highly Capable Large Multimodal Models for Mobile Devices

Machine Reading Comprehension (MRC) reveals the ability to understand a given
text passage and answer questions based on it. Existing research works in MRC
rely heavily on large-size models and corpus to improve the performance
evaluated by metrics such as Exact Match ($EM$) and $F_1$. However, such a
paradigm lacks sufficient interpretation to model capability and can not
efficiently train a model with a large corpus. In this paper, we argue that a
deep understanding of model capabilities and data properties can help us feed a
model with appropriate training data based on its learning status.
Specifically, we design an MRC capability assessment framework that assesses
model capabilities in an explainable and multi-dimensional manner. Based on it,
we further uncover and disentangle the connections between various data
properties and model performance. Finally, to verify the effectiveness of the
proposed MRC capability assessment framework, we incorporate it into a
curriculum learning pipeline and devise a Capability Boundary Breakthrough
Curriculum (CBBC) strategy, which performs a model capability-based training to
maximize the data value and improve training efficiency. Extensive experiments
demonstrate that our approach significantly improves performance, achieving up
to an 11.22% / 8.71% improvement of $EM$ / $F_1$ on MRC tasks.

通过评估模型的能力，并在此基础上基于模型状态提供适当的训练数据，我们设计了一个可解释和多维度的 MRC 模型能力评估框架，并开发了 CBBC 策略，实现了基于模型能力的训练，从而最大化数据价值和提高训练效率。实验表明，我们的方法显著提高了 MRC 任务的性能，$EM/F_1$ 最高可达到 11.22%/8.71%。

通过了解所学来满足所需

Feeding What You Need by Understanding What You Learned

Image super-resolution (SR) is a fast-moving field with novel architectures
attracting the spotlight. However, most SR models were optimized with dated
training strategies. In this work, we revisit the popular RCAN model and
examine the effect of different training options in SR. Surprisingly (or
perhaps as expected), we show that RCAN can outperform or match nearly all the
CNN-based SR architectures published after RCAN on standard benchmarks with a
proper training strategy and minimal architecture change. Besides, although
RCAN is a very large SR architecture with more than four hundred convolutional
layers, we draw a notable conclusion that underfitting is still the main
problem restricting the model capability instead of overfitting. We observe
supportive evidence that increasing training iterations clearly improves the
model performance while applying regularization techniques generally degrades
the predictions. We denote our simply revised RCAN as RCAN-it and recommend
practitioners to use it as baselines for future research. Code is publicly
available at this https URL

该论文重新审视优化 RCAN 模型的不同训练选项对图像超分辨率的影响，指出 RCAN 能够在标准基准上通过适当的训练策略和最小的体系结构修改优化，胜过或匹配几乎所有基于 CNN 的超分辨率架构。此外，该训练发现通过提高训练迭代次数明显提高了模型性能，而应用正则化技术通常会降低预测精度，同时推荐使用简化后的 RCAN 模型作为未来研究的基准模型。

重访 RCAN：图像超分辨率的改进训练

Revisiting RCAN: Improved Training for Image Super-Resolution

Representing a space of handwriting stroke styles includes the challenge of
representing both the style of each character and the overall style of the
human writer. Existing VRNN approaches to representing handwriting often do not
distinguish between these different style components, which can reduce model
capability. Instead, we introduce the Decoupled Style Descriptor (DSD) model
for handwriting, which factors both character- and writer-level styles and
allows our model to represent an overall greater space of styles. This approach
also increases flexibility: given a few examples, we can generate handwriting
in new writer styles, and also now generate handwriting of new characters
across writer styles. In experiments, our generated results were preferred over
a state of the art baseline method 88% of the time, and in a writer
identification task on 20 held-out writers, our DSDs achieved 89.38% accuracy
from a single sample word. Overall, DSDs allows us to improve both the quality
and flexibility over existing handwriting stroke generation approaches.

介绍了 Decoupled Style Descriptor (DSD) 模型，该模型通过因式分解字和笔画风格来提高手写生成的质量和灵活性。在实验中，该模型生成的结果相较于基线方法更受欢迎，并能够很好地完成单样本的作者识别任务。