Leveraging massive knowledge and learning schemes from large language models
(LLMs), recent machine learning models show notable successes in building
generalist agents that exhibit the capability of general-purpose task solving
in diverse domains, including natural language processing, computer vision, and
robotics. However, a significant challenge remains as these models exhibit
limited ability in understanding and interacting with the 3D world. We argue
this limitation significantly hinders the current models from performing
real-world tasks and further achieving general intelligence. To this end, we
introduce an embodied multi-modal and multi-task generalist agent that excels
in perceiving, grounding, reasoning, planning, and acting in the 3D world. Our
proposed agent, referred to as LEO, is trained with shared LLM-based model
architectures, objectives, and weights in two stages: (i) 3D vision-language
alignment and (ii) 3D vision-language-action instruction tuning. To facilitate
the training, we meticulously curate and generate an extensive dataset
comprising object-level and scene-level multi-modal tasks with exceeding scale
and complexity, necessitating a deep understanding of and interaction with the
3D world. Through rigorous experiments, we demonstrate LEO's remarkable
proficiency across a wide spectrum of tasks, including 3D captioning, question
answering, embodied reasoning, embodied navigation, and robotic manipulation.
Our ablation results further provide valuable insights for the development of
future embodied generalist agents.

利用大规模知识和学习方案以及大型语言模型（LLM），最近的机器学习模型在构建通用性代理人方面取得显著成功，表现出在不同领域（包括自然语言处理、计算机视觉和机器人技术）解决通用任务的能力。然而，这些模型在理解和与三维世界交互方面仍存在显著挑战。我们认为这一限制显著阻碍了当前模型执行现实世界任务和进一步实现通用智能的能力。为此，我们提出了一种根植于三维世界中，并具有多模态和多任务的通用代理人，其在感知、根据、推理、规划和行动方面表现出色。我们的提出的代理人，称为 LEO，在两个阶段进行训练：（一）三维视觉语言对齐，（二）三维视觉语言行动指导调整。为了促进训练，我们细致地策划并生成统计规模和复杂性的物体级和场景级多模态任务的大规模数据集，需要对三维世界进行深入的理解和交互。通过严格的实验证明了 LEO 在广泛任务范围中的出色能力，包括三维字幕、问题解答、根植式推理、根植式导航和机器人操作。我们的消融实验结果进一步为未来根植式通用代理的发展提供了有价值的见解。

三维环境中的具身综合应用智能体

An Embodied Generalist Agent in 3D World

Fine-grained financial sentiment analysis on news headlines is a challenging
task requiring human-annotated datasets to achieve high performance. Limited
studies have tried to address the sentiment extraction task in a setting where
multiple entities are present in a news headline. In an effort to further
research in this area, we make publicly available SEntFiN 1.0, a
human-annotated dataset of 10,753 news headlines with entity-sentiment
annotations, of which 2,847 headlines contain multiple entities, often with
conflicting sentiments. We augment our dataset with a database of over 1,000
financial entities and their various representations in news media amounting to
over 5,000 phrases. We propose a framework that enables the extraction of
entity-relevant sentiments using a feature-based approach rather than an
expression-based approach. For sentiment extraction, we utilize 12 different
learning schemes utilizing lexicon-based and pre-trained sentence
representations and five classification approaches. Our experiments indicate
that lexicon-based n-gram ensembles are above par with pre-trained word
embedding schemes such as GloVe. Overall, RoBERTa and finBERT (domain-specific
BERT) achieve the highest average accuracy of 94.29% and F1-score of 93.27%.
Further, using over 210,000 entity-sentiment predictions, we validate the
economic effect of sentiments on aggregate market movements over a long
duration.

本文探讨了在新闻标题中进行细粒度金融情感分析的任务和挑战，提出 SAEntFiN 1.0 数据集，并使用特征为基础的方法提取实体相关情感。使用 12 种不同的学习方案验证实验结果，RoBERTa 和 finBERT 的平均准确率和 F1 分数最高，同时使用 210,000 多个实体情感预测验证了情感对市场的影响。

SEntFiN 1.0：金融新闻情感分析中的实体感知

SEntFiN 1.0: Entity-Aware Sentiment Analysis for Financial News

Consider the following class of learning schemes: $$\hat{\boldsymbol{\beta}}
:= \arg\min_{\boldsymbol{\beta}}\;\sum_{j=1}^n
\ell(\boldsymbol{x}_j^\top\boldsymbol{\beta}; y_j) + \lambda
R(\boldsymbol{\beta}),\qquad\qquad (1) $$ where $\boldsymbol{x}_i \in
\mathbb{R}^p$ and $y_i \in \mathbb{R}$ denote the $i^{\text{th}}$ feature and
response variable respectively. Let $\ell$ and $R$ be the loss function and
regularizer, $\boldsymbol{\beta}$ denote the unknown weights, and $\lambda$ be
a regularization parameter. Finding the optimal choice of $\lambda$ is a
challenging problem in high-dimensional regimes where both $n$ and $p$ are
large. We propose two frameworks to obtain a computationally efficient
approximation ALO of the leave-one-out cross validation (LOOCV) risk for
nonsmooth losses and regularizers. Our two frameworks are based on the primal
and dual formulations of (1). We prove the equivalence of the two approaches
under smoothness conditions. This equivalence enables us to justify the
accuracy of both methods under such conditions. We use our approaches to obtain
a risk estimate for several standard problems, including generalized LASSO,
nuclear norm regularization, and support vector machines. We empirically
demonstrate the effectiveness of our results for non-differentiable cases.

本文提出了两种框架来解决高维情境下通过加入惩罚项来降低过拟合的问题，同时证明了二者在光滑条件下的等效性，并在多个标准问题中验证了该方法的有效性。