Deep Learning (DL) is penetrating into a diverse range of mass mobility,
smart living, and industrial applications, rapidly transforming the way we live
and work. DL is at the heart of many AI implementations. A key set of
challenges is to produce AI modules that are: (1) "circular" - can solve new
tasks without forgetting how to solve previous ones, (2) "secure" - have
immunity to adversarial data attacks, and (3) "tiny" - implementable in low
power low cost embedded hardware. Clearly it is difficult to achieve all three
aspects on a single horizontal layer of platforms, as the techniques require
transformed deep representations that incur different computation and
communication requirements. Here we set out the vision to achieve transformed
DL representations across a 5G and Beyond networked architecture. We first
detail the cross-sectoral motivations for each challenge area, before
demonstrating recent advances in DL research that can achieve circular, secure,
and tiny AI (CST-AI). Recognising the conflicting demand of each transformed
deep representation, we federate their deep learning transformations and
functionalities across the network to achieve connected run-time capabilities.

DL 与 AI 实现之间的关键挑战是实现循环、安全和微小的人工智能模块，该研究提出了在 5G 及其后续网络架构中实现转化的 DL 表示的愿景，并介绍了近期在 DL 研究方面取得的进展。

面向环形、安全和小型人工智能的联邦转换学习

Federated Transformed Learning for a Circular, Secure, and Tiny AI

This study introduces and evaluates tiny, mini, small, and medium-sized
uncased Turkish BERT models, aiming to bridge the research gap in
less-resourced languages. We trained these models on a diverse dataset
encompassing over 75GB of text from multiple sources and tested them on several
tasks, including mask prediction, sentiment analysis, news classification, and,
zero-shot classification. Despite their smaller size, our models exhibited
robust performance, including zero-shot task, while ensuring computational
efficiency and faster execution times. Our findings provide valuable insights
into the development and application of smaller language models, especially in
the context of the Turkish language.

该研究介绍和评估了微小、迷你、小型和中型的非套壳土耳其 BERT 模型，旨在填补资源匮乏语言领域的研究差距。我们使用多个来源的超过 75GB 文本构建了这些模型的训练集，并在多个任务中进行了测试，包括掩码预测、情感分析、新闻分类和零样本分类。尽管模型规模较小，但我们的模型在保证计算效率和更快执行时间的同时，表现出了强大的性能，包括零样本任务。我们的发现对于发展和应用较小语言模型，特别是在土耳其语境下，提供了有价值的见解。