We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code
language model that achieves performance comparable to GPT4-Turbo in
code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained
from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion
tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially
enhances the coding and mathematical reasoning capabilities of DeepSeek-V2,
while maintaining comparable performance in general language tasks. Compared to
DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in
various aspects of code-related tasks, as well as reasoning and general
capabilities. Additionally, DeepSeek-Coder-V2 expands its support for
programming languages from 86 to 338, while extending the context length from
16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves
superior performance compared to closed-source models such as GPT4-Turbo,
Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.

通过深度预训练，DeepSeek-Coder-V2 在编码相关任务方面展示出了显著的先进性，并提高了其推理和总体能力

DeepSeek-Coder-V2: 打破代码智能中闭源模型的限制

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code  Intelligence

We introduce Stable Code, the first in our new-generation of code language
models series, which serves as a general-purpose base code language model
targeting code completion, reasoning, math, and other software
engineering-based tasks. Additionally, we introduce an instruction variant
named Stable Code Instruct that allows conversing with the model in a natural
chat interface for performing question-answering and instruction-based tasks.
In this technical report, we detail the data and training procedure leading to
both models. Their weights are available via Hugging Face for anyone to
download and use at this https URL and
this https URL This report
contains thorough evaluations of the models, including multilingual programming
benchmarks, and the MT benchmark focusing on multi-turn dialogues. At the time
of its release, Stable Code is the state-of-the-art open model under 3B
parameters and even performs comparably to larger models of sizes 7 billion and
15 billion parameters on the popular Multi-PL benchmark. Stable Code Instruct
also exhibits state-of-the-art performance on the MT-Bench coding tasks and on
Multi-PL completion compared to other instruction tuned models. Given its
appealing small size, we also provide throughput measurements on a number of
edge devices. In addition, we open source several quantized checkpoints and
provide their performance metrics compared to the original model.

我们介绍了稳定代码（Stable Code），这是我们新一代的代码语言模型系列中的第一个模型，它用于完成代码补全、推理、数学和其他软件工程任务。此外，我们还引入了一种名为稳定代码指令（Stable Code Instruct）的指令变体，它允许与模型进行自然对话界面以进行问答和基于指令的任务。在这份技术报告中，我们详细介绍了这两个模型的数据和训练过程。他们的权重可通过 Hugging Face 在指定的 URL 下载和使用。该报告对这些模型进行了全面评估，包括多语言编程基准和多轮对话集中在机器翻译方面的基准。稳定代码在发布时是 3B 参数下的最先进的开放模型，甚至在流行的多语言编程基准上，其性能与大小为 70 亿和 150 亿参数的较大模型相当。稳定代码指令模型在 MT-Bench 编码任务和 Multi-PL 代码补全方面也展现了最先进的性能，相较于其他指令调整模型。由于其吸引人的小尺寸，我们还提供了该模型在一些边缘设备上的吞吐量测量。此外，我们还公开了几个量化检查点并提供了它们与原始模型的性能指标对比。

稳定代码技术报告

Stable Code Technical Report

Current generative knowledge graph construction approaches usually fail to
capture structural knowledge by simply flattening natural language into
serialized texts or a specification language. However, large generative
language model trained on structured data such as code has demonstrated
impressive capability in understanding natural language for structural
prediction and reasoning tasks. Intuitively, we address the task of generative
knowledge graph construction with code language model: given a code-format
natural language input, the target is to generate triples which can be
represented as code completion tasks. Specifically, we develop schema-aware
prompts that effectively utilize the semantic structure within the knowledge
graph. As code inherently possesses structure, such as class and function
definitions, it serves as a useful model for prior semantic structural
knowledge. Furthermore, we employ a rationale-enhanced generation method to
boost the performance. Rationales provide intermediate steps, thereby improving
knowledge extraction abilities. Experimental results indicate that the proposed
approach can obtain better performance on benchmark datasets compared with
baselines. Code and datasets are available in
this https URL

通过代码语言模型生成三元组以构建语义结构明确的知识图谱，并使用增强理由生成法提高知识提取能力，实验结果表明该方法在基准数据集上取得优异表现。

CodeKGC: 基于代码的生成式知识图谱构建模型

CodeKGC: Code Language Model for Generative Knowledge Graph Construction

Style is a significant component of natural language text, reflecting a
change in the tone of text while keeping the underlying information the same.
Even though programming languages have strict syntax rules, they also have
style. Code can be written with the same functionality but using different
language features. However, programming style is difficult to quantify, and
thus as part of this work, we define style attributes, specifically for Python.
To build a definition of style, we utilized hierarchical clustering to capture
a style definition without needing to specify transformations. In addition to
defining style, we explore the capability of a pre-trained code language model
to capture information about code style. To do this, we fine-tuned pre-trained
code-language models and evaluated their performance in code style transfer
tasks.

本研究利用 Python 编程语言作为样本，定义了编程代码的风格属性，并探讨了预训练代码语言模型对代码风格的识别能力。我们使用了分层聚类的方法来获取代码风格的定义，通过微调预训练模型，我们还评估了其在代码风格转换中的性能表现。