Regularized inversion methods for image reconstruction are used widely due to
their tractability and ability to combine complex physical sensor models with
useful regularity criteria. Such methods motivated the recently developed
Plug-and-Play prior method, which provides a framework to use advanced
denoising algorithms as regularizers in inversion. However, the need to
formulate regularized inversion as the solution to an optimization problem
limits the possible regularity conditions and physical sensor models.
In this paper, we introduce Consensus Equilibrium (CE), which generalizes
regularized inversion to include a much wider variety of both forward
components and prior components without the need for either to be expressed
with a cost function. CE is based on the solution of a set of equilibrium
equations that balance data fit and regularity. In this framework, the problem
of MAP estimation in regularized inversion is replaced by the problem of
solving these equilibrium equations, which can be approached in multiple ways.
The key contribution of CE is to provide a novel framework for fusing
multiple heterogeneous models of physical sensors or models learned from data.
We describe the derivation of the CE equations and prove that the solution of
the CE equations generalizes the standard MAP estimate under appropriate
circumstances.
We also discuss algorithms for solving the CE equations, including ADMM with
a novel form of preconditioning and Newton's method. We give examples to
illustrate consensus equilibrium and the convergence properties of these
algorithms and demonstrate this method on some toy problems and on a denoising
example in which we use an array of convolutional neural network denoisers,
none of which is tuned to match the noise level in a noisy image but which in
consensus can achieve a better result than any of them individually.

Consensus Equilibrium provides a new framework to fuse a variety of physical sensor models and denoising algorithms without expressing them with a cost function, replacing MAP estimation with solving a set of equilibrium equations that balance data fit and regularity.