As the capabilities of Large Language Models (LLMs) in healthcare and
medicine continue to advance, there is a growing need for competitive
open-source models that can safeguard public interest. With the increasing
availability of highly competitive open base models, the impact of continued
pre-training is increasingly uncertain. In this work, we explore the role of
instruct tuning, model merging, alignment, red teaming and advanced inference
schemes, as means to improve current open models. To that end, we introduce the
Aloe family, a set of open medical LLMs highly competitive within its scale
range. Aloe models are trained on the current best base models (Mistral, LLaMA
3), using a new custom dataset which combines public data sources improved with
synthetic Chain of Thought (CoT). Aloe models undergo an alignment phase,
becoming one of the first few policy-aligned open healthcare LLM using Direct
Preference Optimization, setting a new standard for ethical performance in
healthcare LLMs. Model evaluation expands to include various bias and toxicity
datasets, a dedicated red teaming effort, and a much-needed risk assessment for
healthcare LLMs. Finally, to explore the limits of current LLMs in inference,
we study several advanced prompt engineering strategies to boost performance
across benchmarks, yielding state-of-the-art results for open healthcare 7B
LLMs, unprecedented at this scale.

通过采用指导调优、模型合并、对齐、应用红队技术和先进推理策略来改进当前的开放模型，本文引入了 Aloe 系列医疗领域的开源大型语言模型，成为首批采用直接优化偏好方法进行策略对齐的伦理性能的医疗领域大型语言模型，以及在多个数据集和基准测试中取得了开放医疗领域 7B 语言模型的最新成果。

芦荟：一族经过调优的开放医疗语言模型

Aloe: A Family of Fine-tuned Open Healthcare LLMs

We present a unified framework called deep dependency networks (DDNs) that
combines dependency networks and deep learning architectures for multi-label
classification, with a particular emphasis on image and video data. The primary
advantage of dependency networks is their ease of training, in contrast to
other probabilistic graphical models like Markov networks. In particular, when
combined with deep learning architectures, they provide an intuitive,
easy-to-use loss function for multi-label classification. A drawback of DDNs
compared to Markov networks is their lack of advanced inference schemes,
necessitating the use of Gibbs sampling. To address this challenge, we propose
novel inference schemes based on local search and integer linear programming
for computing the most likely assignment to the labels given observations. We
evaluate our novel methods on three video datasets (Charades, TACoS, Wetlab)
and three image datasets (MS-COCO, PASCAL VOC, NUS-WIDE), comparing their
performance with (a) basic neural architectures and (b) neural architectures
combined with Markov networks equipped with advanced inference and learning
techniques. Our results demonstrate the superiority of our new DDN methods over
the two competing approaches.

我们提出了一个统一的框架，称为深层依赖网络（DDN），它将依赖网络和深度学习架构结合起来，用于多标签分类，特别是在图像和视频数据上。

深度依赖网络和高级推理方案在多标签分类中的应用

Deep Dependency Networks and Advanced Inference Schemes for Multi-Label  Classification

Normalizing flows are machine-learned maps between different lattice theories
which can be used as components in exact sampling and inference schemes.
Ongoing work yields increasingly expressive flows on gauge fields, but it
remains an open question how flows can improve lattice QCD at state-of-the-art
scales. We discuss and demonstrate two applications of flows in replica
exchange (parallel tempering) sampling, aimed at improving topological mixing,
which are viable with iterative improvements upon presently available flows.

通过归一化流在格点理论之间建立机器学习映射，可以作为精确采样和推理方案的组成部分。我们讨论并演示了在复制交换（并行加热）采样中应用流的两个应用，旨在改善拓扑混合，这些应用在当前可用的流的迭代改进基础上是可行的。