Existing frameworks for evaluating and comparing generative models typically
target an offline setting, where the evaluator has access to full batches of
data produced by the models. However, in many practical scenarios, the goal is
to identify the best model using the fewest generated samples to minimize the
costs of querying data from the models. Such an online comparison is
challenging with current offline assessment methods. In this work, we propose
an online evaluation framework to find the generative model that maximizes a
standard assessment score among a group of available models. Our method uses an
optimism-based multi-armed bandit framework to identify the model producing
data with the highest evaluation score, quantifying the quality and diversity
of generated data. Specifically, we study the online assessment of generative
models based on the Fr\'echet Inception Distance (FID) and Inception Score (IS)
metrics and propose the FID-UCB and IS-UCB algorithms leveraging the upper
confidence bound approach in online learning. We prove sub-linear regret bounds
for these algorithms and present numerical results on standard image datasets,
demonstrating their effectiveness in identifying the score-maximizing
generative model.

通过在线评估框架和上置信边界方法，本研究提出了 FID-UCB 和 IS-UCB 算法，用于在线评估生成模型的质量和多样性，证明了这些算法的子线性遗憾界，并在标准图像数据集上呈现了数值实验结果，证明其在识别得分最大化的生成模型方面的有效性。

基于乐观主义的生成模型在线评估方法

An Optimism-based Approach to Online Evaluation of Generative Models

Preschool evaluation is crucial because it gives teachers and parents
influential knowledge about children's growth and development. The COVID-19
pandemic has highlighted the necessity of online assessment for preschool
children. One of the areas that should be tested is their ability to speak.
Employing an Automatic Speech Recognition (ASR) system would not help since
they are pre-trained on voices that differ from children's in terms of
frequency and amplitude. Because most of these are pre-trained with data in a
specific range of amplitude, their objectives do not make them ready for voices
in different amplitudes. To overcome this issue, we added a new objective to
the masking objective of the Wav2Vec 2.0 model called Random Frequency Pitch
(RFP). In addition, we used our newly introduced dataset to fine-tune our model
for Meaningless Words (MW) and Rapid Automatic Naming (RAN) tests. Using
masking in concatenation with RFP outperforms the masking objective of Wav2Vec
2.0 by reaching a Word Error Rate (WER) of 1.35. Our new approach reaches a WER
of 6.45 on the Persian section of the CommonVoice dataset. Furthermore, our
novel methodology produces positive outcomes in zero- and few-shot scenarios.

介绍了一种新的针对学前儿童语音测试的方法，通过加入 Random Frequency Pitch 目标到 Wav2Vec 2.0 模型的屏蔽目标中，在 Persian CommonVoice 数据集上达到了仅有 1.35 的字错误率，并在零次和少次测试中达到了积极的成果。

波斯幼儿语音评估的自动语音识别

Automatic Speech Recognition for Speech Assessment of Persian Preschool Children

With the recent prevalence of remote education, academic assessments are
often conducted online, leading to further concerns surrounding assessment
misconducts. This paper investigates the potentials of online assessment
misconduct (e-cheating) and proposes practical countermeasures against them.
The mechanism for detecting the practices of online cheating is presented in
the form of an e-cheating intelligent agent, comprising of an internet protocol
(IP) detector and a behavioural monitor. The IP detector is an auxiliary
detector which assigns randomised and unique assessment sets as an early
procedure to reduce potential misconducts. The behavioural monitor scans for
irregularities in assessment responses from the candidates, further reducing
any misconduct attempts. This is highlighted through the proposal of the
DenseLSTM using a deep learning approach. Additionally, a new PT Behavioural
Database is presented and made publicly available. Experiments conducted on
this dataset confirm the effectiveness of the DenseLSTM, resulting in
classification accuracies of up to 90.7%.

本文研究了在线考试作弊现象及其潜在危害，并提出了一种基于智能代理和行为监测的措施，包括使用 DenseLSTM 和一个新的 PT 行为数据库，在考试响应中检测和降低不当行为。实验表明，这种方法可达到高达 90.7% 的分类准确率。

使用网络协议和行为分类的在线评估违规检测

Online Assessment Misconduct Detection using Internet Protocol and Behavioural Classification

In the Chinese medical insurance industry, the assessor's role is essential
and requires significant efforts to converse with the claimant. This is a
highly professional job that involves many parts, such as identifying personal
information, collecting related evidence, and making a final insurance report.
Due to the coronavirus (COVID-19) pandemic, the previous offline insurance
assessment has to be conducted online. However, for the junior assessor often
lacking practical experience, it is not easy to quickly handle such a complex
online procedure, yet this is important as the insurance company needs to
decide how much compensation the claimant should receive based on the
assessor's feedback. In order to promote assessors' work efficiency and speed
up the overall procedure, in this paper, we propose a dialogue-based
information extraction system that integrates advanced NLP technologies for
medical insurance assessment. With the assistance of our system, the average
time cost of the procedure is reduced from 55 minutes to 35 minutes, and the
total human resources cost is saved 30% compared with the previous offline
procedure. Until now, the system has already served thousands of online claim
cases.

提出了一种基于对话的信息提取系统，用于医疗保险评估，通过该系统，评估过程的平均时间从 55 分钟缩短到了 35 分钟，总人力成本比以前的线下评估流程节省了 30％。