Evaluating LLM-generated text has become a key challenge, especially in domain-specific contexts like the medical field. This work introduces a novel evaluation methodology for LLM-generated medical explanatory arguments, relying on Proxy Tasks and rankings to closely align results with human evaluation criteria, overcoming the biases typically seen in LLMs used as judges. We demonstrate that the proposed evaluators are robust against adversarial attacks, including the assessment of non-argumentative text. Additionally, the human-crafted arguments needed to train the evaluators are minimized to just one example per Proxy Task. By examining multiple LLM-generated arguments, we establish a methodology for determining whether a Proxy Task is suitable for evaluating LLM-generated medical explanatory arguments, requiring only five examples and two human experts.

本研究解决了在医学领域评估LLM生成文本的关键挑战，通过引入基于代理任务和排名的新评估方法，减少了典型评判中存在的偏见。最重要的发现是，该方法在抵御对抗性攻击方面表现出稳健性，只需最少的训练样本，能有效判断代理任务是否适合评估医学解释性论证。

基于排名而非评分：朝着可靠和稳健的自动评估LLM生成的医学解释性论证