In recent years, great success has been witnessed in building
problem-specific deep networks from unrolling iterative algorithms, for solving
inverse problems and beyond. Unrolling is believed to incorporate the
model-based prior with the learning capacity of deep learning. This paper
revisits the role of unrolling as a design approach for deep networks: to what
extent its resulting special architecture is superior, and can we find better?
Using LISTA for sparse recovery as a representative example, we conduct the
first thorough design space study for the unrolled models. Among all possible
variations, we focus on extensively varying the connectivity patterns and
neuron types, leading to a gigantic design space arising from LISTA. To
efficiently explore this space and identify top performers, we leverage the
emerging tool of neural architecture search (NAS). We carefully examine the
searched top architectures in a number of settings, and are able to discover
networks that are consistently better than LISTA. We further present more
visualization and analysis to "open the black box", and find that the searched
top architectures demonstrate highly consistent and potentially transferable
patterns. We hope our study to spark more reflections and explorations on how
to better mingle model-based optimization prior and data-driven learning.

本文通过对 LISTA 罕见深度网络建立设计空间进行设计空间研究，并利用神经架构搜索技术寻找更优秀的深度网络架构，进行深度学习与模型优化先验结合的探索。