Although the pre-training followed by fine-tuning paradigm is used
extensively in many fields, there is still some controversy surrounding the
impact of pre-training on the fine-tuning process. Currently, experimental
findings based on text and image data lack consensus. To delve deeper into the
unsupervised pre-training followed by fine-tuning paradigm, we have extended
previous research to a new modality: time series. In this study, we conducted a
thorough examination of 150 classification datasets derived from the Univariate
Time Series (UTS) and Multivariate Time Series (MTS) benchmarks. Our analysis
reveals several key conclusions. (i) Pre-training can only help improve the
optimization process for models that fit the data poorly, rather than those
that fit the data well. (ii) Pre-training does not exhibit the effect of
regularization when given sufficient training time. (iii) Pre-training can only
speed up convergence if the model has sufficient ability to fit the data. (iv)
Adding more pre-training data does not improve generalization, but it can
strengthen the advantage of pre-training on the original data volume, such as
faster convergence. (v) While both the pre-training task and the model
structure determine the effectiveness of the paradigm on a given dataset, the
model structure plays a more significant role.

预训练和微调对于数据拟合不佳的模型能够帮助优化过程，但对于拟合较好的模型则没有这种效果；当给予足够的训练时间时，预训练不会表现出正则化的效果；预训练只能加速收敛，前提是模型具备足够的拟合能力；增加更多的预训练数据不能改善泛化能力，但可以增强预训练在原始数据量方面的优势，比如更快的收敛速度；预训练任务和模型结构两者都对于该范式在给定数据集上的效果起到作用，但模型结构的作用更加重要。

探究预训练对时间序列分类的影响

Examining the Effect of Pre-training on Time Series Classification

Recent studies on face attribute editing by exemplars have achieved promising
results due to the increasing power of deep convolutional networks and
generative adversarial networks. These methods encode attribute-related
information in images into the predefined region of the latent feature space by
employing a pair of images with opposite attributes as input to train model,
the face attribute transfer between the input image and the exemplar can be
achieved by exchanging their attribute-related latent feature region. However,
they suffer from three limitations: (1) the model must be trained using a pair
of images with opposite attributes as input; (2) weak capability of editing
multiple attributes by exemplars; (3) poor quality of generating image. Instead
of imposing opposite-attribute constraints on the input image in order to make
the attribute information of images be encoded in the predefined region of the
latent feature space, in this work we directly apply the attribute labels
constraint to the predefined region of the latent feature space. Meanwhile, an
attribute classification loss is employed to make the model learn to extract
the attribute-related information of images into the predefined latent feature
region of the corresponding attribute, which enables our method to transfer
multiple attributes of the exemplar simultaneously. Besides, a novel model
structure is designed to enhance attribute transfer capabilities by exemplars
while improve the quality of the generated image. Experiments demonstrate the
effectiveness of our model on overcoming the above three limitations by
comparing with other methods on the CelebA dataset.

本文中提出的基于属性标签限制的模型，可以通过在预定义的潜在特征空间中直接应用属性标签并使用属性分类损失，从图像中提取属性相关信息，以实现用于编辑多个属性的面部属性传输，同时通过设计新的模型结构来提高属性传输能力。在 CelebA 数据集上的实验结果表明，该方法有效地克服了其他方法存在的三个限制。

MulGAN：基于样例的人脸属性编辑

MulGAN: Facial Attribute Editing by Exemplar

A number of differences have emerged between modern and classic approaches to
constituency parsing in recent years, with structural components like grammars
and feature-rich lexicons becoming less central while recurrent neural network
representations rise in popularity. The goal of this work is to analyze the
extent to which information provided directly by the model structure in
classical systems is still being captured by neural methods. To this end, we
propose a high-performance neural model (92.08 F1 on PTB) that is
representative of recent work and perform a series of investigative
experiments. We find that our model implicitly learns to encode much of the
same information that was explicitly provided by grammars and lexicons in the
past, indicating that this scaffolding can largely be subsumed by powerful
general-purpose neural machinery.

研究了现代和经典的组成分析方法的差异，以及神经网络在此领域的应用。高性能的神经模型代表了最新的工作，实验发现该模型隐式编码了过去语法和词汇表明确提供的信息，暗示这种支撑可以被强大的通用神经机制取代。