We re-examine the problem of reconstructing a high-dimensional signal from a
small set of linear measurements, in combination with image prior from a
diffusion probabilistic model. Well-established methods for optimizing such
measurements include principal component analysis (PCA), independent component
analysis (ICA) and compressed sensing (CS), all of which rely on axis- or
subspace-aligned statistical characterization. But many naturally occurring
signals, including photographic images, contain richer statistical structure.
To exploit such structure, we introduce a general method for obtaining an
optimized set of linear measurements, assuming a Bayesian inverse solution that
leverages the prior implicit in a neural network trained to perform denoising.
We demonstrate that these measurements are distinct from those of PCA and CS,
with significant improvements in minimizing squared reconstruction error. In
addition, we show that optimizing the measurements for the SSIM perceptual loss
leads to perceptually improved reconstruction. Our results highlight the
importance of incorporating the specific statistical regularities of natural
signals when designing effective linear measurements.

从一小组线性测量和来自扩散概率模型的图像先验出发，重新审视了从高维信号重构的问题。通过利用通过神经网络训练进行去噪的先验来获得一组优化的线性测量，不同于主成分分析（PCA）和压缩感知（CS），在最小化均方重构误差方面显著改进。而且，通过最小化结构相似性（SSIM）感知损失来优化测量，可以实现知觉上改进的重构。结果强调了在设计有效的线性测量时，纳入自然信号的特定统计规律的重要性。

基于扩散的图像生成优化线性测量反问题

Optimized Linear Measurements for Inverse Problems using Diffusion-Based  Image Generation

In theoretical cognitive science, there is a tension between highly
structured models whose parameters have a direct psychological interpretation
and highly complex, general-purpose models whose parameters and representations
are difficult to interpret. The former typically provide more insight into
cognition but the latter often perform better. This tension has recently
surfaced in the realm of educational data mining, where a deep learning
approach to predicting students' performance as they work through a series of
exercises---termed deep knowledge tracing or DKT---has demonstrated a stunning
performance advantage over the mainstay of the field, Bayesian knowledge
tracing or BKT. In this article, we attempt to understand the basis for DKT's
advantage by considering the sources of statistical regularity in the data that
DKT can leverage but which BKT cannot. We hypothesize four forms of regularity
that BKT fails to exploit: recency effects, the contextualized trial sequence,
inter-skill similarity, and individual variation in ability. We demonstrate
that when BKT is extended to allow it more flexibility in modeling statistical
regularities---using extensions previously proposed in the literature---BKT
achieves a level of performance indistinguishable from that of DKT. We argue
that while DKT is a powerful, useful, general-purpose framework for modeling
student learning, its gains do not come from the discovery of novel
representations---the fundamental advantage of deep learning. To answer the
question posed in our title, knowledge tracing may be a domain that does not
require `depth'; shallow models like BKT can perform just as well and offer us
greater interpretability and explanatory power.

本文考虑深度知识追踪（DKT）在预测学生学习表现时的优越性及其与贝叶斯知识追踪（BKT）模型的比较，提出了 BKT 模型的拓展方案，最终指出虽然 DKT 是一种实用的学生学习模型，但其对数据的优化不在于发现新的表现形式。