Generative AI offers a simple, prompt-based alternative to fine-tuning
smaller BERT-style LLMs for text classification tasks. This promises to
eliminate the need for manually labeled training data and task-specific model
training. However, it remains an open question whether tools like ChatGPT can
deliver on this promise. In this paper, we show that smaller, fine-tuned LLMs
(still) consistently and significantly outperform larger, zero-shot prompted
models in text classification. We compare three major generative AI models
(ChatGPT with GPT-3.5/GPT-4 and Claude Opus) with several fine-tuned LLMs
across a diverse set of classification tasks (sentiment, approval/disapproval,
emotions, party positions) and text categories (news, tweets, speeches). We
find that fine-tuning with application-specific training data achieves superior
performance in all cases. To make this approach more accessible to a broader
audience, we provide an easy-to-use toolkit alongside this paper. Our toolkit,
accompanied by non-technical step-by-step guidance, enables users to select and
fine-tune BERT-like LLMs for any classification task with minimal technical and
computational effort.

通过比较生成型 AI 模型和经过细调的 LLMs，在文本分类任务中，利用应用特定训练数据进行细调的模型表现优异，从而验证了生成型 AI 模型在兑现其承诺方面存在问题。

微调的「小型」LLM 在文本分类中仍显著优于零样本生成式 AI 模型

Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot  Generative AI Models in Text Classification

Large language models (LLMs) have achieved remarkable success across a wide
spectrum of tasks; however, they still face limitations in scenarios that
demand long-term planning and spatial reasoning. To facilitate this line of
research, in this work, we propose a new benchmark, termed $\textbf{P}$ath
$\textbf{P}$lanning from $\textbf{N}$atural $\textbf{L}$anguage
($\textbf{PPNL}$). Our benchmark evaluates LLMs' spatial-temporal reasoning by
formulating ''path planning'' tasks that require an LLM to navigate to target
locations while avoiding obstacles and adhering to constraints. Leveraging this
benchmark, we systematically investigate LLMs including GPT-4 via different
few-shot prompting methodologies and BART and T5 of various sizes via
fine-tuning. Our experimental results show the promise of few-shot GPT-4 in
spatial reasoning, when it is prompted to reason and act interleavedly,
although it still fails to make long-term temporal reasoning. In contrast,
while fine-tuned LLMs achieved impressive results on in-distribution reasoning
tasks, they struggled to generalize to larger environments or environments with
more obstacles.

大型语言模型在路径规划和空间推理方面取得了显著的成功，少样本的 GPT-4 通过不同的提示方法在空间推理方面表现出了潜力，而微调的 LLMs 则在小环境或有障碍物的情况下难以推广。

大型语言模型能成为良好的路径规划器吗？空间 - 时间推理的基准和研究调查

Can Large Language Models be Good Path Planners? A Benchmark and  Investigation on Spatial-temporal Reasoning

Large language models (LLMs) have shown exceptional performance on a variety
of natural language tasks. Yet, their capabilities for HTML understanding --
i.e., parsing the raw HTML of a webpage, with applications to automation of
web-based tasks, crawling, and browser-assisted retrieval -- have not been
fully explored. We contribute HTML understanding models (fine-tuned LLMs) and
an in-depth analysis of their capabilities under three tasks: (i) Semantic
Classification of HTML elements, (ii) Description Generation for HTML inputs,
and (iii) Autonomous Web Navigation of HTML pages. While previous work has
developed dedicated architectures and training procedures for HTML
understanding, we show that LLMs pretrained on standard natural language
corpora transfer remarkably well to HTML understanding tasks. For instance,
fine-tuned LLMs are 12% more accurate at semantic classification compared to
models trained exclusively on the task dataset. Moreover, when fine-tuned on
data from the MiniWoB benchmark, LLMs successfully complete 50% more tasks
using 192x less data compared to the previous best supervised model. Out of the
LLMs we evaluate, we show evidence that T5-based models are ideal due to their
bidirectional encoder-decoder architecture. To promote further research on LLMs
for HTML understanding, we create and open-source a large-scale HTML dataset
distilled and auto-labeled from CommonCrawl.

通过 fine-tuned LLMs 调整，研究 HTML 理解（元素语义分类，输入描述生成和自主网页导航）任务。 T5-based models 表现最佳，提出一个来自 CommonCrawl 的大规模 HTML 数据集。