In this paper, we implement Neural Ordinary Differential Equations in a
Variational Autoencoder setting for generative time series modeling. An
object-oriented approach to the code was taken to allow for easier development
and research and all code used in the paper can be found here:
this https URL
The results were initially recreated and the reconstructions compared to a
baseline Long-Short Term Memory AutoEncoder. The model was then extended with a
LSTM encoder and challenged by more complex data consisting of time series in
the form of spring oscillations. The model showed promise, and was able to
reconstruct true trajectories for all complexities of data with a smaller RMSE
than the baseline model. However, it was able to capture the dynamic behavior
of the time series for known data in the decoder but was not able to produce
extrapolations following the true trajectory very well for any of the
complexities of spring data. A final experiment was carried out where the model
was also presented with 68 days of solar power production data, and was able to
reconstruct just as well as the baseline, even when very little data is
available.
Finally, the models training time was compared to the baseline. It was found
that for small amounts of data the NODE method was significantly slower at
training than the baseline, while for larger amounts of data the NODE method
would be equal or faster at training.
The paper is ended with a future work section which describes the many
natural extensions to the work presented in this paper, with examples being
investigating further the importance of input data, including extrapolation in
the baseline model or testing more specific model setups.

本文提出了基于神经常微分方程和变分自编码器的生成时序模型，加入 LSTM 编码器并使用多种复杂数据进行了验证；结果表明模型能够有效重构出给定输入并对太阳能生产数据表现良好。但对于复杂数据，其在预测真实轨迹中的表现有限，并建议在未来的研究中探索更多可扩展性。

使用神经常微分方程在变分自编码器中生成时间序列模型

Generative time series models using Neural ODE in Variational Autoencoders

Prepositions are very common and very ambiguous, and understanding their
sense is critical for understanding the meaning of the sentence. Supervised
corpora for the preposition-sense disambiguation task are small, suggesting a
semi-supervised approach to the task. We show that signals from unannotated
multilingual data can be used to improve supervised preposition-sense
disambiguation. Our approach pre-trains an LSTM encoder for predicting the
translation of a preposition, and then incorporates the pre-trained encoder as
a component in a supervised classification system, and fine-tunes it for the
task. The multilingual signals consistently improve results on two
preposition-sense datasets.

本研究采用半监督方法来增强介词意义的自动消歧功能。通过在未标注的多语数据上进行预训练，利用一个 LSTM 编码器来预测介词的翻译，然后在一个监督分类系统中将其作为组成部分进行微调，结果表明这种多语信号能够显著提高两个介词意义数据集的结果表现。