Neuro-Symbolic (NeSy) predictors that conform to symbolic knowledge -
encoding, e.g., safety constraints - can be affected by Reasoning Shortcuts
(RSs): They learn concepts consistent with the symbolic knowledge by exploiting
unintended semantics. RSs compromise reliability and generalization and, as we
show in this paper, they are linked to NeSy models being overconfident about
the predicted concepts. Unfortunately, the only trustworthy mitigation strategy
requires collecting costly dense supervision over the concepts. Rather than
attempting to avoid RSs altogether, we propose to ensure NeSy models are aware
of the semantic ambiguity of the concepts they learn, thus enabling their users
to identify and distrust low-quality concepts. Starting from three simple
desiderata, we derive bears (BE Aware of Reasoning Shortcuts), an ensembling
technique that calibrates the model's concept-level confidence without
compromising prediction accuracy, thus encouraging NeSy architectures to be
uncertain about concepts affected by RSs. We show empirically that bears
improves RS-awareness of several state-of-the-art NeSy models, and also
facilitates acquiring informative dense annotations for mitigation purposes.

通过确保神经符号模型意识到其所学概念的语义歧义，从而使用户能够识别并不信任低质量的概念，从而不降低预测准确性，我们提出了一种名为 BEARS（BE Aware of Reasoning Shortcuts）的合奏技术，该技术校准了模型在概念层面的置信度，提高了对 RS 的感知能力，也有助于获取有信息量的密集注释以用于缓解目的。

BEARS 使神经符号模型意识到其推理捷径

BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts

ChatGPT took the world by storm for its impressive abilities. Due to its
release without documentation, scientists immediately attempted to identify its
limits, mainly through its performance in natural language processing (NLP)
tasks. This paper aims to join the growing literature regarding ChatGPT's
abilities by focusing on its performance in high-resource languages and on its
capacity to predict its answers' accuracy by giving a confidence level. The
analysis of high-resource languages is of interest as studies have shown that
low-resource languages perform worse than English in NLP tasks, but no study so
far has analysed whether high-resource languages perform as well as English.
The analysis of ChatGPT's confidence calibration has not been carried out
before either and is critical to learn about ChatGPT's trustworthiness. In
order to study these two aspects, five high-resource languages and two NLP
tasks were chosen. ChatGPT was asked to perform both tasks in the five
languages and to give a numerical confidence value for each answer. The results
show that all the selected high-resource languages perform similarly and that
ChatGPT does not have a good confidence calibration, often being overconfident
and never giving low confidence values.

ChatGPT 通过在高资源语言中表现和预测答案准确性的能力进行分析，发现高资源语言表现与英语相似，但 ChatGPT 缺乏良好的置信度校准，常常过于自信且不会给出低置信度值。

ChatGPT 提示无法估计高资源语言的预测不确定性

ChatGPT Prompting Cannot Estimate Predictive Uncertainty in  High-Resource Languages

Discriminatively trained, deterministic neural networks are the de facto
choice for classification problems. However, even though they achieve
state-of-the-art results on in-domain test sets, they tend to be overconfident
on out-of-distribution (OOD) data. For instance, ReLU networks -- a popular
class of neural network architectures -- have been shown to almost always yield
high confidence predictions when the test data are far away from the training
set, even when they are trained with OOD data. We overcome this problem by
adding a term to the output of the neural network that corresponds to the logit
of an extra class, that we design to dominate the logits of the original
classes as we move away from the training data.This technique provably prevents
arbitrarily high confidence on far-away test data while maintaining a simple
discriminative point-estimate training. Evaluation on various benchmarks
demonstrates strong performance against competitive baselines on both far-away
and realistic OOD data.

通过在神经网络输出中添加额外类别的逻辑回归值，我们解决了过度自信并提高对边界测试数据的性能表现。