A multi-modal machine learning system uses multiple unique data sources and types to improve its performance. This article proposes a system that combines results from several types of models, all of which are trained on different data signals. As an example to illustrate the efficacy of the system, an experiment is described in which multiple types of data are collected from rats suffering from seizures. This data includes electrocorticography readings, piezoelectric motion sensor data, and video recordings. Separate models are trained on each type of data, with the goal of classifying each time frame as either containing a seizure or not. After each model has generated its classification predictions, these results are combined. While each data signal works adequately on its own for prediction purposes, the significant imbalance in class labels leads to increased numbers of false positives, which can be filtered and removed by utilizing all data sources. This paper will demonstrate that, after postprocessing and combination techniques, classification accuracy is improved with this multi-modal system when compared to the performance of each individual data source.

该研究论文提出了一种多模态机器学习系统，通过结合来自多个独特数据源和类型的结果以提高性能。通过对癫痫大鼠收集的多种类型的数据进行训练和分类，论文表明通过后处理和组合技术，该系统在分类准确性方面优于每个单独数据源的性能。

实验室大鼠自动癫痫检测的多模态机器学习框架