General purpose Large Language Models (LLM) such as the Generative Pretrained
Transformer (GPT) and Large Language Model Meta AI (LLaMA) have attracted much
attention in recent years. There is strong evidence that these models can
perform remarkably well in various natural language processing tasks. However,
how to leverage them to approach domain-specific use cases and drive value
remains an open question. In this work, we focus on a specific use case,
pharmaceutical manufacturing investigations, and propose that leveraging
historical records of manufacturing incidents and deviations in an organization
can be beneficial for addressing and closing new cases, or de-risking new
manufacturing campaigns. Using a small but diverse dataset of real
manufacturing deviations selected from different product lines, we evaluate and
quantify the power of three general purpose LLMs (GPT-3.5, GPT-4, and Claude-2)
in performing tasks related to the above goal. In particular, (1) the ability
of LLMs in automating the process of extracting specific information such as
root cause of a case from unstructured data, as well as (2) the possibility of
identifying similar or related deviations by performing semantic search on the
database of historical records are examined. While our results point to the
high accuracy of GPT-4 and Claude-2 in the information extraction task, we
discuss cases of complex interplay between the apparent reasoning and
hallucination behavior of LLMs as a risk factor. Furthermore, we show that
semantic search on vector embedding of deviation descriptions can be used to
identify similar records, such as those with a similar type of defect, with a
high level of accuracy. We discuss further improvements to enhance the accuracy
of similar record identification.

利用历史记录，将大型语言模型（LLM）应用于制药生产调查，提取特定信息并进行语义搜索。

基础大型语言模型能辅助进行制药生产调查吗？

Can Foundational Large Language Models Assist with Conducting  Pharmaceuticals Manufacturing Investigations?

Historical records of climate fields are often sparse due to missing
measurements, especially before the introduction of large-scale satellite
missions. Several statistical and model-based methods have been introduced to
fill gaps and reconstruct historical records. Here, we employ a recently
introduced deep-learning approach based on Fourier convolutions, trained on
numerical climate model output, to reconstruct historical climate fields. Using
this approach we are able to realistically reconstruct large and irregular
areas of missing data, as well as reconstruct known historical events such as
strong El Ni\~no and La Ni\~na with very little given information. Our method
outperforms the widely used statistical kriging method as well as other recent
machine learning approaches. The model generalizes to higher resolutions than
the ones it was trained on and can be used on a variety of climate fields.
Moreover, it allows inpainting of masks never seen before during the model
training.

通过傅里叶卷积训练的深度学习方法，我们能够重建历史气候场景，包括大片和不规则的缺失数据区域，以及强烈的厄尔尼诺和拉尼娜事件，且效果优于统计克里金方法和其他机器学习方法。该模型可以推广到比其训练分辨率更高的级别，适用于各种气候场景，并允许填补未曾在模型训练中出现的遮罩区域。

用深度学习重建历史气候场

Reconstructing Historical Climate Fields With Deep Learning

Understanding voluminous historical records provides clues on the past in
various aspects, such as social and political issues and even natural science
facts. However, it is generally difficult to fully utilize the historical
records, since most of the documents are not written in a modern language and
part of the contents are damaged over time. As a result, restoring the damaged
or unrecognizable parts as well as translating the records into modern
languages are crucial tasks. In response, we present a multi-task learning
approach to restore and translate historical documents based on a
self-attention mechanism, specifically utilizing two Korean historical records,
ones of the most voluminous historical records in the world. Experimental
results show that our approach significantly improves the accuracy of the
translation task than baselines without multi-task learning. In addition, we
present an in-depth exploratory analysis on our translated results via topic
modeling, uncovering several significant historical events.

本文提出了一种基于自我关注机制的多任务学习方法，用于恢复和翻译历史文献，并通过主题建模进行了深入的探索分析，揭示了多个重要历史事件。