This study addresses key challenges in developing domain-specific large language models (LLMs) for Chinese state-owned assets and enterprises (SOAEs), where current approaches face three limitations: 1) constrained model capacity that limits knowledge integration and cross-task adaptability; 2) excessive reliance on domain-specific supervised fine-tuning (SFT) data, which neglects the broader applicability of general language patterns; and 3) inefficient inference acceleration for large models processing long contexts. In this work, we propose SOAEsV2-7B/72B, a specialized LLM series developed via a three-phase framework: 1) continual pre-training integrates domain knowledge while retaining base capabilities; 2) domain-progressive SFT employs curriculum-based learning strategy, transitioning from weakly relevant conversational data to expert-annotated SOAEs datasets to optimize domain-specific tasks; 3) distillation-enhanced speculative decoding accelerates inference via logit distillation between 72B target and 7B draft models, achieving 1.39-1.52$\times$ speedup without quality loss. Experimental results demonstrate that our domain-specific pre-training phase maintains 99.8% of original general language capabilities while significantly improving domain performance, resulting in a 1.08$\times$ improvement in Rouge-1 score and a 1.17$\times$ enhancement in BLEU-4 score. Ablation studies further show that domain-progressive SFT outperforms single-stage training, achieving 1.02$\times$ improvement in Rouge-1 and 1.06$\times$ in BLEU-4. Our work introduces a comprehensive, full-pipeline approach for optimizing SOAEs LLMs, bridging the gap between general language capabilities and domain-specific expertise.

本研究针对中国国有资产和企业（SOAEs）领域特定大型语言模型（LLMs）开发中的关键挑战，提出了一种三阶段框架，解决当前模型容量受限、过度依赖特定监督数据以及推理加速效率低的问题。实验结果表明，该系列模型在维持总体语言能力的同时，显著提高了领域性能，实现了在Rouge-1和BLEU-4分数上的显著提升，展示了为SOAEs LLMs优化的全面性和有效性。

SOAEsV2-7B/72B: Full-Pipeline Optimization for State-Owned Enterprise
  LLMs via Continual Pre-Training, Domain-Progressive SFT and
  Distillation-Enhanced Speculative Decoding

SOAEsV2-7B/72B：通过持续预训练、领域渐进SFT和蒸馏增强推测解码的国有企业大语言模型全流程优化