While working on a software specification, designers usually need to evaluate
different architectural alternatives to be sure that quality criteria are met.
Even when these quality aspects could be expressed in terms of multiple
software metrics, other qualitative factors cannot be numerically measured, but
they are extracted from the engineer's know-how and prior experiences. In fact,
detecting not only strong but also weak points in the different solutions seems
to fit better with the way humans make their decisions. Putting the human in
the loop brings new challenges to the search-based software engineering field,
especially for those human-centered activities within the early analysis phase.
This paper explores how the interactive evolutionary computation can serve as a
basis for integrating the human's judgment into the search process. An
interactive approach is proposed to discover software architectures, in which
both quantitative and qualitative criteria are applied to guide a
multi-objective evolutionary algorithm. The obtained feedback is incorporated
into the fitness function using architectural preferences allowing the
algorithm to discern between promising and poor solutions. Experimentation with
real users has revealed that the proposed interaction mechanism can effectively
guide the search towards those regions of the search space that are of real
interest to the expert.

通过交互式进化计算以及应用定量和定性评判标准指导多目标进化算法，将用户反馈嵌入到适应函数中，使算法能够辨别出有前景和较差的解决方案，实验证明该交互机制可以有效地引导搜索进入专家真正感兴趣的搜索空间区域。

软件架构的交互式多目标进化优化

Interactive Multi-Objective Evolutionary Optimization of Software  Architectures

There are many ways to express similar things in text, which makes evaluating
natural language generation (NLG) systems difficult. Compounding this
difficulty is the need to assess varying quality criteria depending on the
deployment setting. While the landscape of NLG evaluation has been well-mapped,
practitioners' goals, assumptions, and constraints -- which inform decisions
about what, when, and how to evaluate -- are often partially or implicitly
stated, or not stated at all. Combining a formative semi-structured interview
study of NLG practitioners (N=18) with a survey study of a broader sample of
practitioners (N=61), we surface goals, community practices, assumptions, and
constraints that shape NLG evaluations, examining their implications and how
they embody ethical considerations.

通过对 18 位自然语言生成技术从业者的形成性半结构化访谈和 61 位从业者的调查研究，我们展示了影响自然语言生成技术评估的目标、社区实践、假设和约束，以及它们所体现的伦理考虑。

拆解自然语言生成评估：评估实践、假设及其影响

Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications

While an increasing interest in deep models for single-image depth estimation
methods can be observed, established schemes for their evaluation are still
limited. We propose a set of novel quality criteria, allowing for a more
detailed analysis by focusing on specific characteristics of depth maps. In
particular, we address the preservation of edges and planar regions, depth
consistency, and absolute distance accuracy. In order to employ these metrics
to evaluate and compare state-of-the-art single-image depth estimation
approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera
together with a laser scanner to acquire high-resolution images and highly
accurate depth maps. Experimental results show the validity of our proposed
evaluation protocol.

本文提出了一组新的质量标准，允许更详细地分析深度图的特定特征，以便评估最先进的单图像深度估计方法，并提供了一个新的高质量 RGB-D 数据集。实验结果显示了我们提出的评估协议的有效性。