One of the primary challenges in the field of human activity recognition
(HAR) is the lack of large labeled datasets. This hinders the development of
robust and generalizable models. Recently, cross modality transfer approaches
have been explored that can alleviate the problem of data scarcity. These
approaches convert existing datasets from a source modality, such as video, to
a target modality (IMU). With the emergence of generative AI models such as
large language models (LLMs) and text-driven motion synthesis models, language
has become a promising source data modality as well as shown in proof of
concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of
language-based cross modality transfer to determine their effectiveness for
HAR. Based on this study, we introduce two new extensions for IMUGPT that
enhance its use for practical HAR application scenarios: a motion filter
capable of filtering out irrelevant motion sequences to ensure the relevance of
the generated virtual IMU data, and a set of metrics that measure the diversity
of the generated data facilitating the determination of when to stop generating
virtual IMU data for both effective and efficient processing. We demonstrate
that our diversity metrics can reduce the effort needed for the generation of
virtual IMU data by at least 50%, which open up IMUGPT for practical use cases
beyond a mere proof of concept.

在人类活动识别领域，缺乏大型标记数据集是主要挑战之一。为了解决数据稀缺的问题，最近开始研究跨模态迁移方法，将现有数据集从源模态（如视频）转换为目标模态（IMU）。本研究通过大规模评估语言驱动的跨模态迁移，提出了两个针对实际人类活动识别应用场景的 IMUGPT 扩展，一个是能够滤除无关动作序列以确保生成虚拟 IMU 数据的相关性的动作过滤器，另一个是一套用于测量生成数据多样性的指标，帮助确定何时停止生成虚拟 IMU 数据以实现有效和高效的处理。我们证明了我们的多样性指标可以减少生成虚拟 IMU 数据所需的工作量至少 50％，从而使 IMUGPT 在纯概念证明之外具备实际应用的价值。

IMUGPT 2.0：基于语言的传感器人体活动识别的跨模态转移

IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based  Human Activity Recognition

Modern deep learning heavily relies on large labeled datasets, which often
comse with high costs in terms of both manual labeling and computational
resources. To mitigate these challenges, researchers have explored the use of
informative subset selection techniques, including coreset selection and active
learning. Specifically, coreset selection involves sampling data with both
input ($\bx$) and output ($\by$), active learning focuses solely on the input
data ($\bx$).
In this study, we present a theoretically optimal solution for addressing
both coreset selection and active learning within the context of linear softmax
regression. Our proposed method, COPS (unCertainty based OPtimal Sub-sampling),
is designed to minimize the expected loss of a model trained on subsampled
data. Unlike existing approaches that rely on explicit calculations of the
inverse covariance matrix, which are not easily applicable to deep learning
scenarios, COPS leverages the model's logits to estimate the sampling ratio.
This sampling ratio is closely associated with model uncertainty and can be
effectively applied to deep learning tasks. Furthermore, we address the
challenge of model sensitivity to misspecification by incorporating a
down-weighting approach for low-density samples, drawing inspiration from
previous works.
To assess the effectiveness of our proposed method, we conducted extensive
empirical experiments using deep neural networks on benchmark datasets. The
results consistently showcase the superior performance of COPS compared to
baseline methods, reaffirming its efficacy.

通过开发一个新的算法 COPS，我们提出了一个理论上最佳的解决方案，以最小化从样本数据训练的模型的预期损失。我们的方法在深度学习任务中可以有效地应用，通过使用模型的对数几率来估计采样比例，并通过低密度样本的降权方法来解决模型对错误规范的敏感性挑战。实证实验表明，COPS 相对于基线方法具有卓越的性能。