Looking for sparsity is nowadays crucial to speed up the training of
large-scale neural networks. Projections onto the $\ell_{1,2}$ and
$\ell_{1,\infty}$ are among the most efficient techniques to sparsify and
reduce the overall cost of neural networks. In this paper, we introduce a new
projection algorithm for the $\ell_{1,\infty}$ norm ball. The worst-case time
complexity of this algorithm is $\mathcal{O}\big(nm+J\log(nm)\big)$ for a
matrix in $\mathbb{R}^{n\times m}$. $J$ is a term that tends to 0 when the
sparsity is high, and to $nm$ when the sparsity is low. Its implementation is
easy and it is guaranteed to converge to the exact solution in a finite time.
Moreover, we propose to incorporate the $\ell_{1,\infty}$ ball projection while
training an autoencoder to enforce feature selection and sparsity of the
weights. Sparsification appears in the encoder to primarily do feature
selection due to our application in biology, where only a very small part
($<2\%$) of the data is relevant. We show that both in the biological case and
in the general case of sparsity that our method is the fastest.

寻找稀疏性对于加快大规模神经网络的训练至关重要。本文介绍了一种用于 l1,∞范数球的新投影算法，其最坏情况时间复杂度为 O (nm+Jlog (nm))。我们提出在训练自编码器时将 l1,∞范数球投影结合进去，以强制进行特征选择和稀疏化权重。我们展示了无论是生物学应用中的情况还是一般稀疏性情况下，我们的方法都是最快的。

近线性时间投影到ℓ₁,∞球；稀疏自编码器的应用

Near-Linear Time Projection onto the $\ell_{1,\infty}$ Ball; Application  to Sparse Autoencoders

In this paper, we introduce a neighbor embedding framework for manifold
alignment. We demonstrate the efficacy of the framework using a
manifold-aligned version of the uniform manifold approximation and projection
algorithm. We show that our algorithm can learn an aligned manifold that is
visually competitive to embedding of the whole dataset.

本文介绍了一种用于流形对齐的邻居嵌入框架，并演示了使用经过流形对齐的均匀流形逼近和投影算法学习对齐流形的有效性，该算法可以学习到与整个数据集嵌入可视上相竞争的对齐流形。

流形校准邻居嵌入

Manifold-aligned Neighbor Embedding

A radiograph visualizes the internal anatomy of a patient through the use of
X-ray, which projects 3D information onto a 2D plane. Hence, radiograph
analysis naturally requires physicians to relate the prior about 3D human
anatomy to 2D radiographs. Synthesizing novel radiographic views in a small
range can assist physicians in interpreting anatomy more reliably; however,
radiograph view synthesis is heavily ill-posed, lacking in paired data, and
lacking in differentiable operations to leverage learning-based approaches. To
address these problems, we use Computed Tomography (CT) for radiograph
simulation and design a differentiable projection algorithm, which enables us
to achieve geometrically consistent transformations between the radiography and
CT domains. Our method, XraySyn, can synthesize novel views on real radiographs
through a combination of realistic simulation and finetuning on real
radiographs. To the best of our knowledge, this is the first work on radiograph
view synthesis. We show that by gaining an understanding of radiography in 3D
space, our method can be applied to radiograph bone extraction and suppression
without groundtruth bone labels.

该文介绍了一种基于 CT 投影算法的 X 光图像合成方法，可以在无须地面真值骨标记的情况下从真实的放射性照片中综合出新的视图。

XraySyn：基于 CT 先验的单张 X 光逼真视图合成

XraySyn: Realistic View Synthesis From a Single Radiograph Through CT  Priors

We propose in this paper a general framework for deriving loss functions for
structured prediction. In our framework, the user chooses a convex set
including the output space and provides an oracle for projecting onto that set.
Given that oracle, our framework automatically generates a corresponding convex
and smooth loss function. As we show, adding a projection as output layer
provably makes the loss smaller. We identify the marginal polytope, the output
space's convex hull, as the best convex set on which to project. However,
because the projection onto the marginal polytope can sometimes be expensive to
compute, we allow to use any convex superset instead, with potentially
cheaper-to-compute projection. Since efficient projection algorithms are
available for numerous convex sets, this allows us to construct loss functions
for a variety of tasks. On the theoretical side, when combined with calibrated
decoding, we prove that our loss functions can be used as a consistent
surrogate for a (potentially non-convex) target loss function of interest. We
demonstrate our losses on label ranking, ordinal regression and multilabel
classification, confirming the improved accuracy enabled by projections.

提出一种用于结构化预测的损失函数生成通用框架：用户选择一个凸集，提供一个投影在该集合上的 oracle，框架自动生成一个相应的凸平滑损失函数，并证明在输出层增加投影可以降低损失，通过采用多种凸集实现更多任务，结合校准解码后证明其可以作为（潜在非凸）目标损失函数的一致替身。