This article focuses on the development and evaluation of Small-sized Czech
sentence embedding models. Small models are important components for real-time
industry applications in resource-constrained environments. Given the limited
availability of labeled Czech data, alternative approaches, including
pre-training, knowledge distillation, and unsupervised contrastive fine-tuning,
are investigated. Comprehensive intrinsic and extrinsic analyses are conducted,
showcasing the competitive performance of our models compared to significantly
larger counterparts, with approximately 8 times smaller size and 5 times faster
speed than conventional Base-sized models. To promote cooperation and
reproducibility, both the models and the evaluation pipeline are made publicly
accessible. Ultimately, this article presents practical applications of the
developed sentence embedding models in Seznam.cz, the Czech search engine.
These models have effectively replaced previous counterparts, enhancing the
overall search experience for instance, in organic search, featured snippets,
and image search. This transition has yielded improved performance.

该研究侧重于开发和评估小型捷克句子嵌入模型，通过替代方法进行预训练、知识蒸馏和无监督对比微调来克服标记数据有限的问题，并展示了这些模型在捷克搜索引擎 Seznam.cz 中的实际应用，取得了比常规模型更小 8 倍和更快 5 倍的竞争性性能。

迷你是上品：捷克行业应用语义嵌入模型

Some Like It Small: Czech Semantic Embedding Models for Industry  Applications

Deep learning methods have been successfully applied to remote sensing
problems for several years. Among these methods, CNN based models have high
accuracy in solving the land classification problem using satellite or aerial
images. Although these models have high accuracy, this generally comes with
large memory size requirements. On the other hand, it is desirable to have
small-sized models for applications, such as the ones implemented on unmanned
aerial vehicles, with low memory space. Unfortunately, small-sized CNN models
do not provide high accuracy as with their large-sized versions. In this study,
we propose a novel method to improve the accuracy of CNN models, especially the
ones with small size, by injecting traditional features to them. To test the
effectiveness of the proposed method, we applied it to the CNN models
SqueezeNet, MobileNetV2, ShuffleNetV2, VGG16, and ResNet50V2 having size 0.5 MB
to 528 MB. We used the sample mean, gray level co-occurrence matrix features,
Hu moments, local binary patterns, histogram of oriented gradients, and color
invariants as traditional features for injection. We tested the proposed method
on the EuroSAT dataset to perform land classification. Our experimental results
show that the proposed method significantly improves the land classification
accuracy especially when applied to small-sized CNN models.

本文研究在小型的卷积神经网络（CNN）模型上注入传统特征，以提高陆地分类的准确性，实验结果表明，这种方法可以显著提高使用小型 CNN 模型进行土地分类的准确性。