We introduce LAST, a LAttice-based Speech Transducer library in JAX. With an
emphasis on flexibility, ease-of-use, and scalability, LAST implements
differentiable weighted finite state automaton (WFSA) algorithms needed for
training \& inference that scale to a large WFSA such as a recognition lattice
over the entire utterance. Despite these WFSA algorithms being well-known in
the literature, new challenges arise from performance characteristics of modern
architectures, and from nuances in automatic differentiation. We describe a
suite of generally applicable techniques employed in LAST to address these
challenges, and demonstrate their effectiveness with benchmarks on TPUv3 and
V100 GPU.

介绍了一种基于 Lattice 的 JAX 语音转录库 ——LAST，实现了不同 iable weighted finite state automaton（WFSA）算法，旨在应对 WFSA 算法在现代架构性能和自动微分中的挑战，同时支持 TPUv3 和 V100 GPU 基准测试。

基于 Lattice 的可扩展语音建模在 JAX 中的实现

LAST: Scalable Lattice-Based Speech Modelling in JAX

We present a scalable strategy for development of mesh-free hybrid
neuro-symbolic partial differential equation solvers based on existing
mesh-based numerical discretization methods. Particularly, this strategy can be
used to efficiently train neural network surrogate models for the solution
functions and operators of partial differential equations while retaining the
accuracy and convergence properties of the state-of-the-art numerical solvers.
The presented neural bootstrapping method (hereby dubbed NBM) is based on
evaluation of the finite discretization residuals of the PDE system obtained on
implicit Cartesian cells centered on a set of random collocation points with
respect to trainable parameters of the neural network. We apply NBM to the
important class of elliptic problems with jump conditions across irregular
interfaces in three spatial dimensions. We show the method is convergent such
that model accuracy improves by increasing number of collocation points in the
domain. The algorithms presented here are implemented and released in a
software package named JAX-DIPS (this https URL),
standing for differentiable interfacial PDE solver. JAX-DIPS is purely
developed in JAX, offering end-to-end differentiability from mesh generation to
the higher level discretization abstractions, geometric integrations, and
interpolations, thus facilitating research into use of differentiable
algorithms for developing hybrid PDE solvers.

提出了一种可扩展的方法来开发基于现有网格数字离散化方法的无网格混合神经符号偏微分方程求解器的策略，该策略可用于高效地训练针对部分微分方程的解函数和算子的神经网络替代模型，同时保留了最先进的数字求解器的准确性和收敛性能，从而促进了可微算法用于开发混合 PDE 求解器的研究。

JAX-DIPS: 有限离散方法的神经引导及其在具有间断的椭圆问题中的应用

JAX-DIPS: Neural bootstrapping of finite discretization methods and application to elliptic problems with discontinuities

Classic algorithms and machine learning systems like neural networks are both
abundant in everyday life. While classic computer science algorithms are
suitable for precise execution of exactly defined tasks such as finding the
shortest path in a large graph, neural networks allow learning from data to
predict the most likely answer in more complex tasks such as image
classification, which cannot be reduced to an exact algorithm. To get the best
of both worlds, this thesis explores combining both concepts leading to more
robust, better performing, more interpretable, more computationally efficient,
and more data efficient architectures. The thesis formalizes the idea of
algorithmic supervision, which allows a neural network to learn from or in
conjunction with an algorithm. When integrating an algorithm into a neural
architecture, it is important that the algorithm is differentiable such that
the architecture can be trained end-to-end and gradients can be propagated back
through the algorithm in a meaningful way. To make algorithms differentiable,
this thesis proposes a general method for continuously relaxing algorithms by
perturbing variables and approximating the expectation value in closed form,
i.e., without sampling. In addition, this thesis proposes differentiable
algorithms, such as differentiable sorting networks, differentiable renderers,
and differentiable logic gate networks. Finally, this thesis presents
alternative training strategies for learning with algorithms.

本论文探讨了算法监督的概念，提出了一种连续松弛算法的一般方法，定量评估了算法和神经网络的组合对性能和效率的增益，并探讨了在算法中引入可微性的各种不同技术和训练策略。