Distributions on integers are ubiquitous in probabilistic modeling but remain
challenging for many of today's probabilistic programming languages (PPLs). The
core challenge comes from discrete structure: many of today's PPL inference
strategies rely on enumeration, sampling, or differentiation in order to scale,
which fail for high-dimensional complex discrete distributions involving
integers. Our insight is that there is structure in arithmetic that these
approaches are not using. We present a binary encoding strategy for discrete
distributions that exploits the rich logical structure of integer operations
like summation and comparison. We leverage this structured encoding with
knowledge compilation to perform exact probabilistic inference, and show that
this approach scales to much larger integer distributions with arithmetic.

通过利用整数操作的逻辑结构，我们提出了一种用于离散分布的二进制编码策略，并结合知识编译进行精确的概率推断，从而扩展到更大规模的整数分布。

概率程序中的整数算术扩展

Scaling Integer Arithmetic in Probabilistic Programs

Over the last few years, neural networks have started penetrating safety
critical systems to take decisions in robots, rockets, autonomous driving car,
etc. A problem is that these critical systems often have limited computing
resources. Often, they use the fixed-point arithmetic for its many advantages
(rapidity, compatibility with small memory devices.) In this article, a new
technique is introduced to tune the formats (precision) of already trained
neural networks using fixed-point arithmetic, which can be implemented using
integer operations only. The new optimized neural network computes the output
with fixed-point numbers without modifying the accuracy up to a threshold fixed
by the user. A fixed-point code is synthesized for the new optimized neural
network ensuring the respect of the threshold for any input vector belonging
the range [xmin, xmax] determined during the analysis. From a technical point
of view, we do a preliminary analysis of our floating neural network to
determine the worst cases, then we generate a system of linear constraints
among integer variables that we can solve by linear programming. The solution
of this system is the new fixed-point format of each neuron. The experimental
results obtained show the efficiency of our method which can ensure that the
new fixed-point neural network has the same behavior as the initial
floating-point neural network.

介绍了一种使用定点算数对已训练的神经网络进行精度调整的新技术，其可仅使用整数操作实施，并用线性规划生成一个新的定点格式，以确保其与浮点格式的行为相同，可用于安全关键系统中。

神经网络固定点代码合成

Fixed-Point Code Synthesis For Neural Networks

Standard deep learning algorithms are implemented using floating-point real
numbers. This presents an obstacle for implementing them on low-end devices
which may not have dedicated floating-point units (FPUs). As a result,
researchers in tinyML have considered machine learning algorithms that can
train and run a deep neural network (DNN) on a low-end device using integer
operations only. In this paper we propose PocketNN, a light and self-contained
proof-of-concept framework in pure C++ for the training and inference of DNNs
using only integers. Unlike other approaches, PocketNN directly operates on
integers without requiring any explicit quantization algorithms or customized
fixed-point formats. This was made possible by pocket activations, which are a
family of activation functions devised for integer-only DNNs, and an emerging
DNN training algorithm called direct feedback alignment (DFA). Unlike the
standard backpropagation (BP), DFA trains each layer independently, thus
avoiding integer overflow which is a key problem when using BP with
integer-only operations. We used PocketNN to train some DNNs on two well-known
datasets, MNIST and Fashion-MNIST. Our experiments show that the DNNs trained
with our PocketNN achieved 96.98% and 87.7% accuracies on MNIST and
Fashion-MNIST datasets, respectively. The accuracies are very close to the
equivalent DNNs trained using BP with floating-point real number operations,
such that accuracy degradations were just 1.02%p and 2.09%p, respectively.
Finally, our PocketNN has high compatibility and portability for low-end
devices as it is open source and implemented in pure C++ without any
dependencies.

该研究论文提出了一种使用整数操作进行 DNN 训练和推理的框架 “PocketNN”，并使用 “直接反馈对齐 (DFA)” 算法训练每一层，避免了整数溢出的问题，实验结果表明它在 MNIST 和 Fashion-MNIST 数据集上的准确性与使用 BP 算法相当接近，同时具有高度的兼容性和可移植性。