The limitations of purely neural learning have sparked an interest in
probabilistic neurosymbolic models, which combine neural networks with
probabilistic logical reasoning. As these neurosymbolic models are trained with
gradient descent, we study the complexity of differentiating probabilistic
reasoning. We prove that although approximating these gradients is intractable
in general, it becomes tractable during training. Furthermore, we introduce
WeightME, an unbiased gradient estimator based on model sampling. Under mild
assumptions, WeightME approximates the gradient with probabilistic guarantees
using a logarithmic number of calls to a SAT solver. Lastly, we evaluate the
necessity of these guarantees on the gradient. Our experiments indicate that
the existing biased approximations indeed struggle to optimize even when exact
solving is still feasible.

纯神经学习的局限性引发了对概率神经符号模型的兴趣，这些模型将神经网络与概率逻辑推理相结合。我们研究了不同 iating to withging 行概率推理的复杂性。我们证明了尽管在一般情况下近似这些梯度是棘手的，但在训练过程中变得可行。此外，我们介绍了基于模型采样的无偏梯度估计器 WeightME。在温和的假设下，WeightME 用对数数量的 SAT 求解器调用来近似梯度并提供概率保证。最后，我们评估了梯度中这些保证的必要性。我们的实验表明，即使在仍然可行的情况下，现有的偏倚近似确实很难优化。

关于概率神经符号学习的难度

On the Hardness of Probabilistic Neurosymbolic Learning

Deep neural networks have emerged as the workhorse for a large section of
robotics and control applications, especially as models for dynamical systems.
Such data-driven models are in turn used for designing and verifying autonomous
systems. This is particularly useful in modeling medical systems where data can
be leveraged to individualize treatment. In safety-critical applications, it is
important that the data-driven model is conformant to established knowledge
from the natural sciences. Such knowledge is often available or can often be
distilled into a (possibly black-box) model $M$. For instance, the unicycle
model (which encodes Newton's laws) for an F1 racing car. In this light, we
consider the following problem - given a model $M$ and state transition
dataset, we wish to best approximate the system model while being bounded
distance away from $M$. We propose a method to guarantee this conformance. Our
first step is to distill the dataset into few representative samples called
memories, using the idea of a growing neural gas. Next, using these memories we
partition the state space into disjoint subsets and compute bounds that should
be respected by the neural network, when the input is drawn from a particular
subset. This serves as a symbolic wrapper for guaranteed conformance. We argue
theoretically that this only leads to bounded increase in approximation error;
which can be controlled by increasing the number of memories. We experimentally
show that on three case studies (Car Model, Drones, and Artificial Pancreas),
our constrained neurosymbolic models conform to specified $M$ models (each
encoding various constraints) with order-of-magnitude improvements compared to
the augmented Lagrangian and vanilla training methods. Our code can be found at
this https URL

本研究提出了一种基于神经符号模型的方法，可以实现对模型的确定性接纳，并在三项案例中进行了实验验证。

神经符号模型对自然限制的保证一致性

Guaranteed Conformance of Neurosymbolic Models to Natural Constraints

Sequence-to-sequence (seq2seq) models have been successful across many NLP
tasks, including ones that require predicting linguistic structure. However,
recent work on compositional generalization has shown that seq2seq models
achieve very low accuracy in generalizing to linguistic structures that were
not seen in training. We present new evidence that this is a general limitation
of seq2seq models that is present not just in semantic parsing, but also in
syntactic parsing and in text-to-text tasks, and that this limitation can often
be overcome by neurosymbolic models that have linguistic knowledge built in. We
further report on some experiments that give initial answers on the reasons for
these limitations.

通过实验证明，Seq2Seq 模型在英语句法分析、语义分析和文本之间的转换任务中，普遍存在推广的普适性较低的问题，但是在建立了语言知识的神经符号模型中，这些限制往往可以克服。