Used car pricing is a critical aspect of the automotive industry, influenced
by many economic factors and market dynamics. With the recent surge in online
marketplaces and increased demand for used cars, accurate pricing would benefit
both buyers and sellers by ensuring fair transactions. However, the transition
towards automated pricing algorithms using machine learning necessitates the
comprehension of model uncertainties, specifically the ability to flag
predictions that the model is unsure about. Although recent literature proposes
the use of boosting algorithms or nearest neighbor-based approaches for swift
and precise price predictions, encapsulating model uncertainties with such
algorithms presents a complex challenge. We introduce ProbSAINT, a model that
offers a principled approach for uncertainty quantification of its price
predictions, along with accurate point predictions that are comparable to
state-of-the-art boosting techniques. Furthermore, acknowledging that the
business prefers pricing used cars based on the number of days the vehicle was
listed for sale, we show how ProbSAINT can be used as a dynamic forecasting
model for predicting price probabilities for different expected offer duration.
Our experiments further indicate that ProbSAINT is especially accurate on
instances where it is highly certain. This proves the applicability of its
probabilistic predictions in real-world scenarios where trustworthiness is
crucial.

用车定价受经济因素和市场动态的影响，自动化定价算法的不确定性在应用中面临复杂挑战，ProbSAINT 模型提供了准确定价和不确定性量化的方法，适用于预测不同预期交易期限的价格概率。

ProbSAINT：用于二手车定价的概率表格回归

ProbSAINT: Probabilistic Tabular Regression for Used Car Pricing

Robust reinforcement learning (RL) aims at learning a policy that optimizes
the worst-case performance over an uncertainty set. Given nominal Markov
decision process (N-MDP) that generates samples for training, the set contains
MDPs obtained by some perturbations from N-MDP. In this paper, we introduce a
new uncertainty set containing more realistic MDPs in practice than the
existing sets. Using this uncertainty set, we present a robust RL, named
ARQ-Learning, for tabular cases. Also, we characterize the finite-time error
bounds and prove that it converges as fast as Q-Learning and robust Q-Learning
(i.e., the state-of-the-art robust RL method) while providing better robustness
for real applications. We propose {\em pessimistic agent} that efficiently
tackles the key bottleneck for the extension of ARQ-Learning into large or
continuous state spaces. Using this technique, we first propose PRQ-Learning.
To the next, combining this with DQN and DDPG, we develop PR-DQN and PR-DDPG,
respectively. We emphasize that our technique can be easily combined with the
other popular model-free methods. Via experiments, we demonstrate the
superiority of the proposed methods in various RL applications with model
uncertainties.

介绍了一种新的不确定性集合并基于此提出了一种名为 ARQ-Learning 的鲁棒强化学习方法，同时还提出一种能高效解决 ARQ-Learning 在大规模或连续状态空间下的问题的技术，最终将其应用于各种存在模型不确定性的强化学习应用中。

实用鲁棒强化学习：邻域不确定性集和双代理算法

On practical robust reinforcement learning: adjacent uncertainty set and  double-agent algorithm

In this paper, we focus on the problem of robustifying reinforcement learning
(RL) algorithms with respect to model uncertainties. Indeed, in the framework
of model-based RL, we propose to merge the theory of constrained Markov
decision process (CMDP), with the theory of robust Markov decision process
(RMDP), leading to a formulation of robust constrained-MDPs (RCMDP). This
formulation, simple in essence, allows us to design RL algorithms that are
robust in performance, and provides constraint satisfaction guarantees, with
respect to uncertainties in the system's states transition probabilities. The
need for RCMPDs is important for real-life applications of RL. For instance,
such formulation can play an important role for policy transfer from simulation
to real world (Sim2Real) in safety critical applications, which would benefit
from performance and safety guarantees which are robust w.r.t model
uncertainty. We first propose the general problem formulation under the concept
of RCMDP, and then propose a Lagrangian formulation of the optimal problem,
leading to a robust-constrained policy gradient RL algorithm. We finally
validate this concept on the inventory management problem.

本文介绍了一个基于 Constrained Markov Decision Process（CMDP）和 Robust Markov Decision Process（RMDP）的框架，即 Robust Constrained-MDPs（RCMDP），用于设计强大而稳健的强化学习算法，并提供相应的约束满足保证。同时，还将这个框架用于从模拟到真实世界的政策转移中，以实现对模型不确定性的强鲁棒性和安全保障。最后，我们在库存管理问题上验证了这个框架的有效性。