Deep cascaded architectures for magnetic resonance imaging (MRI) acceleration
have shown remarkable success in providing high-quality reconstruction.
However, as the number of cascades increases, the improvements in
reconstruction tend to become marginal, indicating possible excess model
capacity. Knowledge distillation (KD) is an emerging technique to compress
these models, in which a trained deep teacher network is used to distill
knowledge to a smaller student network such that the student learns to mimic
the behavior of the teacher. Most KD methods focus on effectively training the
student with a pre-trained teacher unaware of the student model. We propose
SFT-KD-Recon, a student-friendly teacher training approach along with the
student as a prior step to KD to make the teacher aware of the structure and
capacity of the student and enable aligning the representations of the teacher
with the student. In SFT, the teacher is jointly trained with the unfolded
branch configurations of the student blocks using three loss terms -
teacher-reconstruction loss, student-reconstruction loss, and teacher-student
imitation loss, followed by KD of the student. We perform extensive experiments
for MRI acceleration in 4x and 5x under-sampling on the brain and cardiac
datasets on five KD methods using the proposed approach as a prior step. We
consider the DC-CNN architecture and setup teacher as D5C5 (141765 parameters),
and student as D3C5 (49285 parameters), denoting a compression of 2.87:1.
Results show that (i) our approach consistently improves the KD methods with
improved reconstruction performance and image quality, and (ii) the student
distilled using our approach is competitive with the teacher, with the
performance gap reduced from 0.53 dB to 0.03 dB.

深度级联体系结构、知识蒸馏、MRI 加速、SFT-KD-Recon 和重建性能。