Large Language Models (LLMs) have become integral components in various
autonomous agent systems. In this study, we present an exploration-based
trajectory optimization approach, referred to as ETO. This learning method is
designed to enhance the performance of open LLM agents. Contrary to previous
studies that exclusively train on successful expert trajectories, our method
allows agents to learn from their exploration failures. This leads to improved
performance through an iterative optimization framework. During the exploration
phase, the agent interacts with the environment while completing given tasks,
gathering failure trajectories to create contrastive trajectory pairs. In the
subsequent training phase, the agent utilizes these trajectory preference pairs
to update its policy using contrastive learning methods like DPO. This
iterative cycle of exploration and training fosters continued improvement in
the agents. Our experiments on three complex tasks demonstrate that ETO
consistently surpasses baseline performance by a large margin. Furthermore, an
examination of task-solving efficiency and potential in scenarios lacking
expert trajectory underscores the effectiveness of our approach.

大型语言模型已成为各种自主代理系统中不可或缺的组成部分，在本研究中我们提出了一种基于探索的轨迹优化方法，称为 ETO，通过允许代理从探索失败中学习而提高开放式 LLM 代理的性能，不同于之前只对成功专家轨迹进行训练的方法，我们的方法允许代理通过对比学习方法（如 DPO）利用这些轨迹偏好对来更新政策，通过探索和训练的迭代循环促进了代理的持续改进，在三个复杂任务上的实验表明 ETO 始终以较大幅度超过基线性能，并且在缺乏专家轨迹的情景中展示了其解任务效率和潜力的有效性。

试错法：基于探索的轨迹优化方法用于 LLM 代理

Trial and Error: Exploration-Based Trajectory Optimization for LLM  Agents

In recent years, the explosion of web videos makes text-video retrieval
increasingly essential and popular for video filtering, recommendation, and
search. Text-video retrieval aims to rank relevant text/video higher than
irrelevant ones. The core of this task is to precisely measure the cross-modal
similarity between texts and videos. Recently, contrastive learning methods
have shown promising results for text-video retrieval, most of which focus on
the construction of positive and negative pairs to learn text and video
representations. Nevertheless, they do not pay enough attention to hard
negative pairs and lack the ability to model different levels of semantic
similarity. To address these two issues, this paper improves contrastive
learning using two novel techniques. First, to exploit hard examples for robust
discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module
(DMAE) to mine hard negative pairs from textual and visual clues. By further
introducing a Negative-aware InfoNCE (NegNCE) loss, we are able to adaptively
identify all these hard negatives and explicitly highlight their impacts in the
training loss. Second, our work argues that triplet samples can better model
fine-grained semantic similarity compared to pairwise samples. We thereby
present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to
construct partial order triplet samples by automatically generating
fine-grained hard negatives for matched text-video pairs. The proposed TPM-CL
designs an adaptive token masking strategy with cross-modal interaction to
model subtle semantic differences. Extensive experiments demonstrate that the
proposed approach outperforms existing methods on four widely-used text-video
retrieval datasets, including MSR-VTT, MSVD, DiDeMo and ActivityNet.

通过改进对比学习方法，引入新的技术，分别利用文本和视觉线索，从中挖掘出困难负样例，并且能够自适应地确定它们对训练损失的影响；同时，通过构建部分有序三元组样本来模拟细粒度语义相似性，以提高文本 - 视频检索的性能。

双模态注意力增强的文本 - 视频检索与三元局部对比学习

Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial  Margin Contrastive Learning

Unsupervised representation learning approaches aim to learn discriminative
feature representations from unlabeled data, without the requirement of
annotating every sample. Enabling unsupervised representation learning is
extremely crucial for time series data, due to its unique annotation bottleneck
caused by its complex characteristics and lack of visual cues compared with
other data modalities. In recent years, unsupervised representation learning
techniques have advanced rapidly in various domains. However, there is a lack
of systematic analysis of unsupervised representation learning approaches for
time series. To fill the gap, we conduct a comprehensive literature review of
existing rapidly evolving unsupervised representation learning approaches for
time series. Moreover, we also develop a unified and standardized library,
named ULTS (i.e., Unsupervised Learning for Time Series), to facilitate fast
implementations and unified evaluations on various models. With ULTS, we
empirically evaluate state-of-the-art approaches, especially the rapidly
evolving contrastive learning methods, on 9 diverse real-world datasets. We
further discuss practical considerations as well as open research challenges on
unsupervised representation learning for time series to facilitate future
research in this field.

无监督时间序列数据表示学习方法的文献综述及开放性研究挑战的综合分析，通过统一标准化库对其进行了实证评估，并讨论了实际考虑因素和未来研究方向。