Significant strides have been made in natural language tasks, largely
attributed to the emergence of powerful large language models (LLMs). These
models, pre-trained on extensive and diverse corpora, have become increasingly
capable of comprehending the intricacies of language. Despite the abundance of
LLMs for many high-resource languages, the availability of such models remains
limited for European Portuguese. We introduce Gl\'orIA, a robust European
Portuguese decoder LLM. To pre-train Gl\'orIA, we assembled a comprehensive
PT-PT text corpus comprising 35 billion tokens from various sources. We present
our pre-training methodology, followed by an assessment of the model's
effectiveness on multiple downstream tasks. Additionally, to evaluate our
models' language modeling capabilities, we introduce CALAME-PT (Context-Aware
LAnguage Modeling Evaluation for Portuguese), the first Portuguese zero-shot
language-modeling benchmark. Evaluation shows that Gl\'orIA significantly
outperforms existing open PT decoder models in language modeling and that it
can generate sound, knowledge-rich, and coherent PT-PT text. The model also
exhibits strong potential for various downstream tasks.

通过 Gl'orIA 这一强大的欧洲葡萄牙语解码器 LLM，在多个下游任务中，Gl'orIA 在语言模型和生成有丰富知识且连贯的葡萄牙语文本方面远优于现有的 PT 解码模型。