Pre-trained language models like ChatGPT have significantly improved code
generation. As these models scale up, there is an increasing need for the
output to handle more intricate tasks. Moreover, in bioinformatics, generating
functional programs poses additional notable challenges due to the amount of
domain knowledge, the need for complicated data operations, and intricate
functional dependencies between the operations. Here, we present BioCoder, a
benchmark developed to evaluate existing pre-trained models in generating
bioinformatics code. In relation to function-code generation, BioCoder covers
potential package dependencies, class declarations, and global variables. It
incorporates 1026 functions and 1243 methods in Python and Java from GitHub and
253 examples from the Rosalind Project. BioCoder incorporates a fuzz-testing
framework for evaluation, and we have applied it to evaluate many models
including InCoder, CodeGen, CodeGen2, SantaCoder, StarCoder, StarCoder+,
InstructCodeT5+, and ChatGPT. Our detailed analysis of these models emphasizes
the importance of domain knowledge, pragmatic code generation, and contextual
understanding. Our dataset, benchmark, Docker images, and scripts required for
testing are all available at this https URL

基于预训练语言模型的生物信息学代码生成的基准评估 BioCoder 表明，领域知识、实际代码生成和上下文理解的重要性。

生物编码器：基于上下文语用知识的生物信息学代码生成基准

BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual  Pragmatic Knowledge

Program synthesis is the process of generating a computer program following a
set of specifications, which can be a high-level description of the problem
and/or a set of input-output examples. The synthesis can be modeled as a search
problem in which the search space is the set of all the programs valid under a
grammar. As the search space is vast, brute force is usually not viable and
search heuristics, such as genetic programming, also have difficulty navigating
it without any guidance. In this paper we present HOTGP, a new genetic
programming algorithm that synthesizes pure, typed, and functional programs.
HOTGP leverages the knowledge provided by the rich data-types associated with
the specification and the built-in grammar to constrain the search space and
improve the performance of the synthesis. The grammar is based on Haskell's
standard base library (the synthesized code can be directly compiled using any
standard Haskell compiler) and includes support for higher-order functions,
$\lambda$-functions, and parametric polymorphism. Experimental results show
that, when compared to $6$ state-of-the-art algorithms using a standard set of
benchmarks, HOTGP is competitive and capable of synthesizing the correct
programs more frequently than any other of the evaluated algorithms.

本文介绍了一种新的基于 Haskell 的基因编程算法 HOTGP，它利用规约的数据类型来约束搜索空间，更好地合成类型良好的纯函数编程。经验结果表明，与其他六种算法相比，HOTGP 在综合正确程序的能力上更为优异。

HOTGP -- 高阶类型遗传规划

HOTGP -- Higher-Order Typed Genetic Programming

We introduce GQA, a new dataset for real-world visual reasoning and
compositional question answering, seeking to address key shortcomings of
previous VQA datasets. We have developed a strong and robust question engine
that leverages scene graph structures to create 22M diverse reasoning
questions, all come with functional programs that represent their semantics. We
use the programs to gain tight control over the answer distribution and present
a new tunable smoothing technique to mitigate question biases. Accompanying the
dataset is a suite of new metrics that evaluate essential qualities such as
consistency, grounding and plausibility. An extensive analysis is performed for
baselines as well as state-of-the-art models, providing fine-grained results
for different question types and topologies. Whereas a blind LSTM obtains mere
42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%,
offering ample opportunity for new research to explore. We strongly hope GQA
will provide an enabling resource for the next generation of models with
enhanced robustness, improved consistency, and deeper semantic understanding
for images and language.

GQA 是一个旨在解决以往 VQA 数据集的缺陷问题的实际可视化推理和组成问题回答数据集，它使用场景图结构创建了 2200 万个不同的推理问题，伴随着功能程序共同表示它们的语义，其中包括新的可调平滑技术来缓解问题偏差。贯穿数据集的还有一套新的度量标准，可以评估基本的质量，例如一致性、基础和可信性。

GQA：一个适用于真实世界视觉推理和组合问答的新数据集

GQA: A New Dataset for Real-World Visual Reasoning and Compositional  Question Answering

Explaining and reasoning about processes which underlie observed black-box
phenomena enables the discovery of causal mechanisms, derivation of suitable
abstract representations and the formulation of more robust predictions. We
propose to learn high level functional programs in order to represent abstract
models which capture the invariant structure in the observed data. We introduce
the $\pi$-machine (program-induction machine) -- an architecture able to induce
interpretable LISP-like programs from observed data traces. We propose an
optimisation procedure for program learning based on backpropagation, gradient
descent and A* search. We apply the proposed method to two problems: system
identification of dynamical systems and explaining the behaviour of a DQN
agent. Our results show that the $\pi$-machine can efficiently induce
interpretable programs from individual data traces.

该研究提出了一种可以从观察到的数据中感知先验结构并产生可解释程序的优化过程，适用于系统识别和行为解释。