Machine programming (MP) is an emerging field at the intersection of
deterministic and probabilistic computing, and it aims to assist software and
hardware engineers, among other applications. Along with powerful compute
resources, MP systems often rely on vast amount of open-source code to learn
interesting properties about code and programming and solve problems in the
areas of debugging, code recommendation, auto-completion, etc. Unfortunately,
several of the existing MP systems either do not consider quality of code
repositories or use atypical quality measures than those typically used in
software engineering community to select them. As such, impact of quality of
code repositories on the performance of these systems needs to be studied.
In this preliminary paper, we evaluate impact of different quality
repositories on the performance of a candidate MP system. Towards that
objective, we develop a framework, named GitRank, to rank open-source
repositories on quality, maintainability, and popularity by leveraging existing
research on this topic. We then apply GitRank to evaluate correlation between
the quality measures used by the candidate MP system and the quality measures
used by our framework. Our preliminary results reveal some correlation between
the quality measures used in GitRank and ControlFlag's performance, suggesting
that some of the measures used in GitRank are applicable to ControlFlag. But it
also raises questions around right quality measures for code repositories used
in MP systems. We believe that our findings also generate interesting insights
towards code quality measures that affect performance of MP systems.

本文研究机器编程中代码库质量对性能的影响，提出了一个排名框架 GitRank，并将其应用于评估候选 MP 系统使用的质量措施和 GitRank 使用的质量措施之间的相关性。初步结果表明 GitRank 衡量的质量措施可适用于 ControlFlag，但同时也引发了有关 MP 系统使用的代码库质量措施的问题。

机器编程系统是否使用正确的源代码度量来选择代码库？

Are Machine Programming Systems using Right Source-Code Measures to Select Code Repositories?

Open-source repositories provide wealth of information and are increasingly
being used to build artificial intelligence (AI) based systems to solve
problems in software engineering. Open-source repositories could be of varying
quality levels, and bad-quality repositories could degrade performance of these
systems. Evaluating quality of open-source repositories, which is not available
directly on code hosting sites such as GitHub, is thus important. In this
hackathon, we utilize known code quality measures and GrimoireLab toolkit to
implement a framework, named GitRank, to rank open-source repositories on three
different criteria. We discuss our findings and preliminary evaluation in this
hackathon report.

本文介绍一种名为 GitRank 的框架，利用已知的代码质量度量方法和 GrimoireLab 工具包的排名系统，对开源代码库进行三个不同标准的排名，以评估开源代码库的质量。

GitRank: 一个排名 GitHub 存储库的框架

GitRank: A Framework to Rank GitHub Repositories

Code retrieval techniques and tools have been playing a key role in
facilitating software developers to retrieve existing code fragments from
available open-source repositories given a user query. Despite the existing
efforts in improving the effectiveness of code retrieval, there are still two
main issues hindering them from being used to accurately retrieve satisfiable
code fragments from large-scale repositories when answering complicated
queries. First, the existing approaches only consider shallow features of
source code such as method names and code tokens, but ignoring structured
features such as abstract syntax trees (ASTs) and control-flow graphs (CFGs) of
source code, which contains rich and well-defined semantics of source code.
Second, although the deep learning-based approach performs well on the
representation of source code, it lacks the explainability, making it hard to
interpret the retrieval results and almost impossible to understand which
features of source code contribute more to the final results.
To tackle the two aforementioned issues, this paper proposes MMAN, a novel
Multi-Modal Attention Network for semantic source code retrieval. A
comprehensive multi-modal representation is developed for representing
unstructured and structured features of source code, with one LSTM for the
sequential tokens of code, a Tree-LSTM for the AST of code and a GGNN (Gated
Graph Neural Network) for the CFG of code. Furthermore, a multi-modal attention
fusion layer is applied to assign weights to different parts of each modality
of source code and then integrate them into a single hybrid representation.
Comprehensive experiments and analysis on a large-scale real-world dataset show
that our proposed model can accurately retrieve code snippets and outperforms
the state-of-the-art methods.

本文提出了一种新的多模式注意力网络（MMAN）来解决代码检索过程中的两个主要问题：缺少结构化特征以及深度学习方法的可解释性，其中.MMAM 能够准确地从大规模的代码库中检索代码片段，同时在实际数据集上也比当前最先进的方法表现更优秀。