Small Language Models (SLMs) are generally considered to be more compact
versions of large language models (LLMs), typically having fewer than 7 billion
parameters. This study investigates the ability of small language models to
learn, retain, and subsequently eliminate noise that is typically not found on
the internet, where most pretraining datasets are sourced. For this, four
pre-trained SLMs were utilized: Olmo 1B, Qwen1.5 1.8B, Gemma 2B, and Phi2 2.7B.
The models were instruction-tuned without noise and tested for task execution
with in-context learning. Afterward, noise patterns were introduced to evaluate
the models' learning and unlearning capabilities. We evaluated the models'
performance at various training levels. Phi consistently excelled with
word-level noise but performed the worst with character-level noise. Despite
being the smallest with approximately 1 billion parameters, Olmo performed
consistently well on tasks.

该研究调查了小语言模型在学习、保留和消除通常不存在于互联网上的噪声方面的能力，并评估了四个小语言模型的表现在不同训练水平下，发现 Olmo 在任务上表现始终良好，即使它是参数最少的模型。

小型语言模型能否学习、遗忘并保留噪声模式？

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?

Spectrum analysis systems in online water quality testing are designed to
detect types and concentrations of pollutants and enable regulatory agencies to
respond promptly to pollution incidents. However, spectral data-based testing
devices suffer from complex noise patterns when deployed in non-laboratory
environments. To make the analysis model applicable to more environments, we
propose a noise patterns transferring model, which takes the spectrum of
standard water samples in different environments as cases and learns the
differences in their noise patterns, thus enabling noise patterns to transfer
to unknown samples. Unfortunately, the inevitable sample-level baseline noise
makes the model unable to obtain the paired data that only differ in
dataset-level environmental noise. To address the problem, we generate a
sample-to-sample case-base to exclude the interference of sample-level noise on
dataset-level noise learning, enhancing the system's learning performance.
Experiments on spectral data with different background noises demonstrate the
good noise-transferring ability of the proposed method against baseline systems
ranging from wavelet denoising, deep neural networks, and generative models.
From this research, we posit that our method can enhance the performance of DL
models by generating high-quality cases. The source code is made publicly
available online at this https URL

为了使光谱分析模型能够适用于更多环境，我们提出了一种噪声模式传递模型，通过学习不同环境下标准水样本的噪声模式差异，使噪声模式能够传递到未知样本中，进而增强系统的学习性能。通过实验证明，我们的方法能够提高深度学习模型的性能，生成高质量的案例。

我们能够传输噪声模式吗？使用生成的案例的多环境频谱分析模型

Can We Transfer Noise Patterns? An Multi-environment Spectrum Analysis  Model Using Generated Cases

Using printed photograph and replaying videos of biometric modalities, such
as iris, fingerprint and face, are common attacks to fool the recognition
systems for granting access as the genuine user. With the growing online
person-to-person shopping (e.g., Ebay and Craigslist), such attacks also
threaten those services, where the online photo illustration might not be
captured from real items but from paper or digital screen. Thus, the study of
anti-spoofing should be extended from modality-specific solutions to
generic-object-based ones. In this work, we define and tackle the problem of
Generic Object Anti-Spoofing (GOAS) for the first time. One significant cue to
detect these attacks is the noise patterns introduced by the capture sensors
and spoof mediums. Different sensor/medium combinations can result in diverse
noise patterns. We propose a GAN-based architecture to synthesize and identify
the noise patterns from seen and unseen medium/sensor combinations. We show
that the procedure of synthesis and identification are mutually beneficial. We
further demonstrate the learned GOAS models can directly contribute to
modality-specific anti-spoofing without domain transfer. The code and GOSet
dataset are available at cvlab.cse.msu.edu/project-goas.html.

本研究定义并解决了首个泛对象反欺骗问题（GOAS），并提出了基于 GAN 的体系结构来合成和识别不同传感器 / 介质组合引入的噪声模式，它对于材料规范的反欺骗起到直接的贡献。