Solving partial differential equations (PDEs) has been a fundamental problem
in computational science and of wide applications for both scientific and
engineering research. Due to its universal approximation property, neural
network is widely used to approximate the solutions of PDEs. However, existing
works are incapable of solving high-order PDEs due to insufficient calculation
accuracy of higher-order derivatives, and the final network is a black box
without explicit explanation. To address these issues, we propose a deep
learning framework to solve high-order PDEs, named SHoP. Specifically, we
derive the high-order derivative rule for neural network, to get the
derivatives quickly and accurately; moreover, we expand the network into a
Taylor series, providing an explicit solution for the PDEs. We conduct
experimental validations four high-order PDEs with different dimensions,
showing that we can solve high-order PDEs efficiently and accurately.

本文提出了一种名为 SHoP 的深度学习框架，通过推导神经网络的高阶导数规则和将网络扩展为泰勒级数，实现了高阶偏微分方程的快速准确求解，并通过实验验证了其有效性。

SHoP：用于求解高阶偏微分方程的深度学习框架

SHoP: A Deep Learning Framework for Solving High-order Partial  Differential Equations

The learning process of deep learning methods usually updates the model's
parameters in multiple iterations. Each iteration can be viewed as the
first-order approximation of Taylor's series expansion. The remainder, which
consists of higher-order terms, is usually ignored in the learning process for
simplicity. This learning scheme empowers various multimedia based
applications, such as image retrieval, recommendation system, and video search.
Generally, multimedia data (e.g., images) are semantics-rich and
high-dimensional, hence the remainders of approximations are possibly non-zero.
In this work, we consider the remainder to be informative and study how it
affects the learning process. To this end, we propose a new learning approach,
namely gradient adjustment learning (GAL), to leverage the knowledge learned
from the past training iterations to adjust vanilla gradients, such that the
remainders are minimized and the approximations are improved. The proposed GAL
is model- and optimizer-agnostic, and is easy to adapt to the standard learning
framework. It is evaluated on three tasks, i.e., image classification, object
detection, and regression, with state-of-the-art models and optimizers. The
experiments show that the proposed GAL consistently enhances the evaluated
models, whereas the ablation studies validate various aspects of the proposed
GAL. The code is available at
https://github.com/luoyan407/gradient_adjustment.git.

本文提出了一种名为梯度调整学习的新学习方法，旨在通过调整梯度降低近似中的余项，并改善近似质量。实验结果表明，该方法可提升现有模型的性能表现。

有监督学习中最小化剩余误差的学习

Learning to Minimize the Remainder in Supervised Learning

Benefiting from the contiguous representation ability, deep implicit
functions can extract the iso-surface of a shape at arbitrary resolution.
However, utilizing the neural network with a large number of parameters as the
implicit function prevents the generation speed of high-resolution topology
because it needs to forward a large number of query points into the network. In
this work, we propose TaylorImNet inspired by the Taylor series for implicit 3D
shape representation. TaylorImNet exploits a set of discrete expansion points
and corresponding Taylor series to model a contiguous implicit shape field.
After the expansion points and corresponding coefficients are obtained, our
model only needs to calculate the Taylor series to evaluate each point and the
number of expansion points is independent of the generating resolution. Based
on this representation, our TaylorImNet can achieve a significantly faster
generation speed than other baselines. We evaluate our approach on
reconstruction tasks from various types of input, and the experimental results
demonstrate that our approach can get slightly better performance than existing
state-of-the-art baselines while improving the inference speed with a large
margin.

本研究提出使用 Taylor 级数的 TaylorImNet 模型来拟合连续隐式函数，该模型可以在不牺牲高分辨率的情况下提高 3D 形状生成的速度和精度。