Convolutional neural networks (CNNs) have gained widespread usage across
various fields such as weather forecasting, computer vision, autonomous
driving, and medical image analysis due to its exceptional ability to extract
spatial information, share parameters, and learn local features. However, the
practical implementation and commercialization of CNNs in these domains are
hindered by challenges related to model sizes, overfitting, and computational
time. To address these limitations, our study proposes a groundbreaking
approach that involves tensorizing the dense layers in the CNN to reduce model
size and computational time. Additionally, we incorporate attention layers into
the CNN and train it using Contrastive self-supervised learning to effectively
classify cloud information, which is crucial for accurate weather forecasting.
We elucidate the key characteristics of tensorized neural network (TNN),
including the data compression rate, accuracy, and computational speed. The
results indicate how TNN change their properties under the batch size setting.

该研究提出了一种具有突破性的方法，通过对卷积神经网络中的密集层进行张量化，来减小模型大小和计算时间，同时结合注意力层和对比自监督学习训练以有效分类云信息，从而能够准确进行天气预测。

基于张量化神经网络的云分类应用

Application of Tensorized Neural Networks for Cloud Classification

We present an electrocardiogram (ECG) -based emotion recognition system using
self-supervised learning. Our proposed architecture consists of two main
networks, a signal transformation recognition network and an emotion
recognition network. First, unlabelled data are used to successfully train the
former network to detect specific pre-determined signal transformations in the
self-supervised learning step. Next, the weights of the convolutional layers of
this network are transferred to the emotion recognition network, and two dense
layers are trained in order to classify arousal and valence scores. We show
that our self-supervised approach helps the model learn the ECG feature
manifold required for emotion recognition, performing equal or better than the
fully-supervised version of the model. Our proposed method outperforms the
state-of-the-art in ECG-based emotion recognition with two publicly available
datasets, SWELL and AMIGOS. Further analysis highlights the advantage of our
self-supervised approach in requiring significantly less data to achieve
acceptable results.

使用自我监督学习的心电图基于情感识别系统在分类唤醒度和价值评分方面表现出色，并且使用未标记的数据帮助该模型学习心电图特征流形。

基于自监督学习的基于心电图的情感识别

Self-supervised Learning for ECG-based Emotion Recognition

In response to the development of recent efficient dense layers, this paper
shows that something as simple as replacing linear components in pointwise
convolutions with structured linear decompositions also produces substantial
gains in the efficiency/accuracy tradeoff. Pointwise convolutions are fully
connected layers and are thus prepared for replacement by structured
transforms. Networks using such layers are able to learn the same tasks as
those using standard convolutions, and provide Pareto-optimal benefits in
efficiency/accuracy, both in terms of computation (mult-adds) and parameter
count (and hence memory). Code is available at
this https URL

本文表明，使用结构化线性分解替换点积卷积中的线性元素不仅更高效，而且还提供了在效率 / 准确性方面的达成帕累托最优的成果。