3D content creation from text prompts has shown remarkable success recently.
However, current text-to-3D methods often generate 3D results that do not align
well with human preferences. In this paper, we present a comprehensive
framework, coined DreamReward, to learn and improve text-to-3D models from
human preference feedback. To begin with, we collect 25k expert comparisons
based on a systematic annotation pipeline including rating and ranking. Then,
we build Reward3D -- the first general-purpose text-to-3D human preference
reward model to effectively encode human preferences. Building upon the 3D
reward model, we finally perform theoretical analysis and present the Reward3D
Feedback Learning (DreamFL), a direct tuning algorithm to optimize the
multi-view diffusion models with a redefined scorer. Grounded by theoretical
proof and extensive experiment comparisons, our DreamReward successfully
generates high-fidelity and 3D consistent results with significant boosts in
prompt alignment with human intention. Our results demonstrate the great
potential for learning from human feedback to improve text-to-3D models.

从人类偏好反馈中学习和改进文本生成 3D 模型的综合框架，通过理论分析和广泛实验比较，我们的 DreamReward 成功生成高保真和与人类意图一致的 3D 结果，展示了从人类反馈中改进文本生成 3D 模型的巨大潜力。