In the current landscape of large language models (LLMs), the process of
instruction tuning serves as an essential step. Considering the high computing
power overhead, data-efficient instruction tuning was proposed to reduce the
training data size in this process, aiming at selecting high-quality
instructional data. Nevertheless, we argue that most current data-efficient
instruction-tuning methods are highly dependent on the quality of the original
instruction-tuning dataset. When it comes to datasets synthesized by LLMs, a
common scenario in this field, dirty samples will even be selected with a
higher probability than other samples. To address these challenges, we utilized
external knowledge (relevant examples or paragraphs) to evaluate those samples
synthesized by LLMs with an in-context-based relative predictive entropy. Based
on the new metric, we proposed a framework, dubbed as \textbf{RECOST}, which
integrates external-knowledge-base re-ranking and diversity-consistent sampling
into a single pipeline. Through extensive experiments on several synthetic
datasets (Alpaca and Alpaca-gpt4), we demonstrate the effectiveness of our
method and achieve even better results with only \textbf{1\%} of the full
dataset.

使用外部知识与基于上下文的相对预测熵评估由大型语言模型合成的样本，并结合多样性一致性抽样，提出了一种名为 RECOST 的框架，通过在合成数据集中实验展示了方法的有效性，并仅使用全数据集的 1％即可获得更好的结果。

RECOST: 基于外部知识引导的数据高效训练指导优化

RECOST: External Knowledge Guided Data-efficient Instruction Tuning

Recently, removing objects from videos and filling in the erased regions
using deep video inpainting (VI) algorithms has attracted considerable
attention. Usually, a video sequence and object segmentation masks for all
frames are required as the input for this task. However, in real-world
applications, providing segmentation masks for all frames is quite difficult
and inefficient. Therefore, we deal with VI in a one-shot manner, which only
takes the initial frame's object mask as its input. Although we can achieve
that using naive combinations of video object segmentation (VOS) and VI
methods, they are sub-optimal and generally cause critical errors. To address
that, we propose a unified pipeline for one-shot video inpainting (OSVI). By
jointly learning mask prediction and video completion in an end-to-end manner,
the results can be optimal for the entire task instead of each separate module.
Additionally, unlike the two stage methods that use the predicted masks as
ground truth cues, our method is more reliable because the predicted masks can
be used as the network's internal guidance. On the synthesized datasets for
OSVI, our proposed method outperforms all others both quantitatively and
qualitatively.

本研究提出了一种统一的方法来实现一次性视频修复，它可以同时进行遮罩预测和视频完成，并通过端到端的学习来优化整个任务，其效果在合成数据集上表现最佳。

单张视频修复

One-Shot Video Inpainting

Recently, there has been increasing interest in synthesizing data to improve
downstream text-to-SQL tasks. In this paper, we first examined the existing
synthesized datasets and discovered that state-of-the-art text-to-SQL
algorithms did not further improve on popular benchmarks when trained with
augmented synthetic data. We observed two shortcomings: illogical synthetic SQL
queries from independent column sampling and arbitrary table joins. To address
these issues, we propose a novel synthesis framework that incorporates key
relationships from schema, imposes strong typing, and conducts
schema-distance-weighted column sampling. We also adopt an intermediate
representation (IR) for the SQL-to-text task to further improve the quality of
the generated natural language questions. When existing powerful semantic
parsers are pre-finetuned on our high-quality synthesized data, our experiments
show that these models have significant accuracy boosts on popular benchmarks,
including new state-of-the-art performance on Spider.

本文提出了一种新颖的数据合成框架，其中包括来自模式的关键关系、强类型和基于模式距离的列抽样，并采用了 SQL-to-text 任务的中间表示（IR），进一步提高了生成的自然语言问题的质量。实验证明，当现有强大的语义解析器在高质量的合成数据上进行预调整时，这些模型在流行的基准测试中的准确性得到了显着提高。

文本到 SQL 解析中合成高质量数据的重要性

Importance of Synthesizing High-quality Data for Text-to-SQL Parsing

Shadow removal is an essential task for scene understanding. Many studies
consider only matching the image contents, which often causes two types of
ghosts: color in-consistencies in shadow regions or artifacts on shadow
boundaries. In this paper, we tackle these issues in two ways. First, to
carefully learn the border artifacts-free image, we propose a novel network
structure named the dual hierarchically aggregation network~(DHAN). It contains
a series of growth dilated convolutions as the backbone without any
down-samplings, and we hierarchically aggregate multi-context features for
attention and prediction, respectively. Second, we argue that training on a
limited dataset restricts the textural understanding of the network, which
leads to the shadow region color in-consistencies. Currently, the largest
dataset contains 2k+ shadow/shadow-free image pairs. However, it has only 0.1k+
unique scenes since many samples share exactly the same background with
different shadow positions. Thus, we design a shadow matting generative
adversarial network~(SMGAN) to synthesize realistic shadow mattings from a
given shadow mask and shadow-free image. With the help of novel masks or
scenes, we enhance the current datasets using synthesized shadow images.
Experiments show that our DHAN can erase the shadows and produce high-quality
ghost-free images. After training on the synthesized and real datasets, our
network outperforms other state-of-the-art methods by a large margin. The code
is available: this http URL

本文介绍了通过使用名为 DHAN 和 SMGAN 的网络结构来解决阴影区域颜色不一致和边界失真的问题，以提高场景理解的能力。