Stochastic optimization algorithms have been successfully applied in several
domains to find optimal solutions. Because of the ever-growing complexity of
the integrated systems, novel stochastic algorithms are being proposed, which
makes the task of the performance analysis of the algorithms extremely
important. In this paper, we provide a novel ranking scheme to rank the
algorithms over multiple single-objective optimization problems. The results of
the algorithms are compared using a robust bootstrapping-based hypothesis
testing procedure that is based on the principles of severity. Analogous to the
football league scoring scheme, we propose pairwise comparison of algorithms as
in league competition. Each algorithm accumulates points and a performance
metric of how good or bad it performed against other algorithms analogous to
goal differences metric in football league scoring system. The goal differences
performance metric can not only be used as a tie-breaker but also be used to
obtain a quantitative performance of each algorithm. The key novelty of the
proposed ranking scheme is that it takes into account the performance of each
algorithm considering the magnitude of the achieved performance improvement
along with its practical relevance and does not have any distributional
assumptions. The proposed ranking scheme is compared to classical hypothesis
testing and the analysis of the results shows that the results are comparable
and our proposed ranking showcases many additional benefits.

提出了一种新的排名方案，利用基于鲁棒的自助法假设检验程序对多个单目标优化问题的算法进行排名，并考虑了算法的性能改进的大小和实际相关性，与传统假设检验相比，提出的排名方案具有可比性和许多额外的好处。

一种基于严重程度原则的针对随机优化算法性能分析的新型排名方案

A Novel Ranking Scheme for the Performance Analysis of Stochastic  Optimization Algorithms using the Principles of Severity

The quality of non-default ranking on e-commerce platforms, such as based on
ascending item price or descending historical sales volume, often suffers from
acute relevance problems, since the irrelevant items are much easier to be
exposed at the top of the ranking results. In this work, we propose a two-stage
ranking scheme, which first recalls wide range of candidate items through
refined query/title keyword matching, and then classifies the recalled items
using BERT-Large fine-tuned on human label data. We also implemented parallel
prediction on multiple GPU hosts and a C++ tokenization custom op of
Tensorflow. In this data challenge, our model won the 1st place in the
supervised phase (based on overall F1 score) and 2nd place in the final phase
(based on average per query F1 score).

本文提出了一种基于 Bert-Large 的双阶段排序方案，首先通过精细的查询 / 标题关键词匹配检索候选商品，然后利用经过人工标注数据的 BERT-Large 进行分类，同时在多个 GPU 主机上实现并行预测和基于 Tensorflow 的 C++ 分词自定义操作。在数据挑战中，我们的模型在监督阶段的总 F1 分数上获得了第一名，在最终阶段的平均每个查询的 F1 分数上获得了第二名。

针对电子商务非默认搜索排名的 BERT 微调

Fine-tune BERT for E-commerce Non-Default Search Ranking

In this paper, we propose a novel online topic tracking framework, named
IEDL, for tracking the topic changes related to deep learning techniques on
Stack Exchange and automatically interpreting each identified topic. The
proposed framework combines the prior topic distributions in a time window
during inferring the topics in current time slice, and introduces a new ranking
scheme to select most representative phrases and sentences for the inferred
topics in each time slice. Experiments on 7,076 Stack Exchange posts show the
effectiveness of IEDL in tracking topic changes and labeling topics.

本研究提出了一种名为 IEDL 的新型在线主题跟踪框架，用于跟踪与深度学习技术相关的主题变化并自动解释每个识别出的主题。该框架在推断当前时间片段中的主题时结合了时间窗口中的先前主题分布，并引入了一种新的排名方案，以选择最具代表性的短语和句子。针对 7,076 个 Stack Exchange 帖子的实验证明，IEDL 在跟踪主题变化和标记主题方面非常有效。