In this study, we investigate the enhancement of the GPT Neo 125M performance
in Community Question Answering (CQA) with a focus on programming, through the
integration of Reinforcement Learning from Human Feedback (RLHF) and the
utilization of scores from Stack Overflow. Two distinct reward model training
strategies are employed for fine-tuning with Proximal Policy Optimization
(PPO). Notably, the improvements in performance achieved through this method
are comparable to those of GPT Neo 2.7B parameter variant. Additionally, an
auxiliary scoring mechanism is introduced, which demonstrates the limitations
of conventional linguistic metrics in evaluating responses in the programming
domain. Through accurate analysis, this paper looks at the divergence between
traditional linguistic metrics and our human-preferences-based reward model,
underscoring the imperative for domain-specific evaluation methods. By
elucidating the complexities involved in applying RLHF to programming CQA and
accentuating the significance of context-aware evaluation, this study
contributes to the ongoing efforts in refining Large Language Models through
focused human feedback.

本研究通过采用人类反馈驱动的强化学习与来自 Stack Overflow 的得分，加强了 GPT Neo 125M 在编程社区问答（CQA）中的性能，在使用近端策略优化（PPO）的 fine-tuning 过程中使用了两种不同的奖励模型训练策略，并通过引入辅助评分机制，揭示了在编程领域中常规语言度量在评估回答时的局限性，强调了领域特定评估方法的重要性，通过准确的分析，本文探讨了将人类反馈驱动的强化学习应用于编程 CQA 以及环境感知评估的复杂性，在优化大型语言模型方面对持续努力做出了贡献。

公共社区评分用于编程领域中的问答强化学习

Reinforcement learning for question answering in programming domain  using public community scoring as a human feedback

In this work, we address the problem of assessing and constructing feedback
for early-stage writing automatically using machine learning. Early-stage
writing is typically vastly different from conventional writing due to phonetic
spelling and lack of proper grammar, punctuation, spacing etc. Consequently,
early-stage writing is highly non-trivial to analyze using common linguistic
metrics. We propose to use sequence-to-sequence models for "translating"
early-stage writing by students into "conventional" writing, which allows the
translated text to be analyzed using linguistic metrics. Furthermore, we
propose a novel robust likelihood to mitigate the effect of noise in the
dataset. We investigate the proposed methods using a set of numerical
experiments and demonstrate that the conventional text can be predicted with
high accuracy.

使用机器学习自动评估和构建早期写作的反馈，通过序列到序列模型将学生的早期写作 “翻译” 成 “传统” 写作，从而能够使用语言度量对翻译后的文本进行分析，并使用鲁棒似然方法减轻数据集中的噪声效应，并通过一系列数值实验证明能以高准确率预测传统文本。

基于神经网络的儿童早期写作自动反馈

Neural machine translation for automated feedback on children's  early-stage writing

People vary in their ability to make accurate predictions about the future.
Prior studies have shown that some individuals can predict the outcome of
future events with consistently better accuracy. This leads to a natural
question: what makes some forecasters better than others? In this paper we
explore connections between the language people use to describe their
predictions and their forecasting skill. Datasets from two different
forecasting domains are explored: (1) geopolitical forecasts from Good Judgment
Open, an online prediction forum and (2) a corpus of company earnings forecasts
made by financial analysts. We present a number of linguistic metrics which are
computed over text associated with people's predictions about the future
including: uncertainty, readability, and emotion. By studying linguistic
factors associated with predictions, we are able to shed some light on the
approach taken by skilled forecasters. Furthermore, we demonstrate that it is
possible to accurately predict forecasting skill using a model that is based
solely on language. This could potentially be useful for identifying accurate
predictions or potentially skilled forecasters earlier.

本文探讨了语言因素与预测技能之间的联系，并基于语言模型对预测技能进行了准确预测。

从文本中测量预测技能

Measuring Forecasting Skill from Text

We propose the inverse problem of Visual question answering (iVQA), and
explore its suitability as a benchmark for visuo-linguistic understanding. The
iVQA task is to generate a question that corresponds to a given image and
answer pair. Since the answers are less informative than the questions, and the
questions have less learnable bias, an iVQA model needs to better understand
the image to be successful than a VQA model. We pose question generation as a
multi-modal dynamic inference process and propose an iVQA model that can
gradually adjust its focus of attention guided by both a partially generated
question and the answer. For evaluation, apart from existing linguistic
metrics, we propose a new ranking metric. This metric compares the ground truth
question's rank among a list of distractors, which allows the drawbacks of
different algorithms and sources of error to be studied. Experimental results
show that our model can generate diverse, grammatically correct and content
correlated questions that match the given answer.

本文提出了逆向视觉问答（iVQA）问题，并探讨其作为视觉语言理解基准的适用性。我们提出了一种 iVQA 模型，可以逐渐调整其注意力焦点，以便更好地理解给定的图像。实验结果表明，我们的模型可以生成多样性、语法正确且内容相关的问题，与给定的答案相匹配。