A recent method employs 3D voxels to represent 3D shapes, but this limits the
approach to low resolutions due to the computational cost caused by the cubic
complexity of 3D voxels. Hence the method suffers from a lack of detailed
geometry. To resolve this issue, we propose Y^2Seq2Seq, a view-based model, to
learn cross-modal representations by joint reconstruction and prediction of
view and word sequences. Specifically, the network architecture of Y^2Seq2Seq
bridges the semantic meaning embedded in the two modalities by two coupled `Y'
like sequence-to-sequence (Seq2Seq) structures. In addition, our novel
hierarchical constraints further increase the discriminability of the
cross-modal representations by employing more detailed discriminative
information. Experimental results on cross-modal retrieval and 3D shape
captioning show that Y^2Seq2Seq outperforms the state-of-the-art methods.

本文提出了一种基于视角和词序列联合重构和预测的交叉模态表示 Y^2Seq2Seq 模型，采用新颖层次约束的方法增加交叉模态表示的辨别力，并在交叉模态检索和 3D 形状字幕生成上实现了更好的性能表现。

Y^2Seq2Seq：联合重建和预测视图和单词序列的三维形状与文本的跨模态表示学习

Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by  Joint Reconstruction and Prediction of View and Word Sequences

Knowledge Transfer (KT) techniques tackle the problem of transferring the
knowledge from a large and complex neural network into a smaller and faster
one. However, existing KT methods are tailored towards classification tasks and
they cannot be used efficiently for other representation learning tasks. In
this paper a novel knowledge transfer technique, that is capable of training a
student model that maintains the same amount of mutual information between the
learned representation and a set of (possible unknown) labels as the teacher
model, is proposed. Apart from outperforming existing KT techniques, the
proposed method allows for overcoming several limitations of existing methods
providing new insight into KT as well as novel KT applications, ranging from
knowledge transfer from handcrafted feature extractors to {cross-modal} KT from
the textual modality into the representation extracted from the visual modality
of the data.

本文提出了一种新的知识迁移技术，能够训练一个学生模型，使其保持与教师模型之间相同数量的互信息，与现有的 KT 方法相比，该方法能够克服一些局限性，并提供新的 KT 应用。

利用概率知识传递学习深度表示

Learning Deep Representations with Probabilistic Knowledge Transfer

People can recognize scenes across many different modalities beyond natural
images. In this paper, we investigate how to learn cross-modal scene
representations that transfer across modalities. To study this problem, we
introduce a new cross-modal scene dataset. While convolutional neural networks
can categorize cross-modal scenes well, they also learn an intermediate
representation not aligned across modalities, which is undesirable for
cross-modal transfer applications. We present methods to regularize cross-modal
convolutional neural networks so that they have a shared representation that is
agnostic of the modality. Our experiments suggest that our scene representation
can help transfer representations across modalities for retrieval. Moreover,
our visualizations suggest that units emerge in the shared representation that
tend to activate on consistent concepts independently of the modality.

本文介绍了一种新的跨模态场景数据集，并研究了如何学习跨模态场景的表示。通过正则化跨模态卷积神经网络，我们提出了一种共享表示方法，可以帮助跨模态检索表征。此外，我们的可视化结果表明，共享表示中出现了单元，它们倾向于独立于模态激活的一致概念。