Prior knowledge and symbolic rules in machine learning are often expressed in the form of label constraints, especially in structured prediction problems. In this work, we compare two common strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference, by quantifying their impact on model performance. For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints. However, its preference for small violations introduces a bias toward a suboptimal model. For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage. Given these differences, we further explore the use of two approaches together and propose conditions for constrained inference to compensate for the bias introduced by regularization, aiming to improve both the model complexity and optimal risk.

在机器学习中，先验知识和符号规则通常以标签约束的形式表达，本文比较了两种常见的将标签约束编码到机器学习流程中的策略，即带约束的正则化和约束推断，并通过量化它们对模型性能的影响。通过正则化，作者表明它通过排除与约束不一致的模型来缩小泛化差距，但其偏好小违规限制了模型朝着最优模型的方向发展。另一方面，通过约束推断，作者展示了它通过修正模型的违规行为来降低总体风险，并将违规行为转化为优势。鉴于这些差异，作者进一步探索了同时使用两种方法的可能性，并提出了约束推断条件以弥补正则化引入偏差的问题，以提高模型的复杂度和最优风险。

关于标签约束的正则化和推理