Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of
thousands of vehicles in a city to millions of ride demands throughout the day,
providing great promises for improving transportation efficiency through the
tasks of order dispatching and vehicle repositioning. Existing studies,
however, usually consider the two tasks in simplified settings that hardly
address the complex interactions between the two, the real-time fluctuations
between supply and demand, and the necessary coordinations due to the
large-scale nature of the problem. In this paper we propose a unified
value-based dynamic learning framework (V1D3) for tackling both tasks. At the
center of the framework is a globally shared value function that is updated
continuously using online experiences generated from real-time platform
transactions. To improve the sample-efficiency and the robustness, we further
propose a novel periodic ensemble method combining the fast online learning
with a large-scale offline training scheme that leverages the abundant
historical driver trajectory data. This allows the proposed framework to adapt
quickly to the highly dynamic environment, to generalize robustly to recurrent
patterns and to drive implicit coordinations among the population of managed
vehicles. Extensive experiments based on real-world datasets show considerably
improvements over other recently proposed methods on both tasks. Particularly,
V1D3 outperforms the first prize winners of both dispatching and repositioning
tracks in the KDD Cup 2020 RL competition, achieving state-of-the-art results
on improving both total driver income and user experience related metrics.

本文提出了一个名为 V1D3 的 value-based 动态学习框架，它可以同时处理车辆分配和重新定位问题，并结合在线体验和历史轨迹数据进行周期性合成，取得了大幅提升，成为 KDD Cup 2020 RL 竞赛中车辆调度和定位的冠军。

价值函数是你所需的一切：适用于出行平台的统一学习框架

Value Function is All You Need: A Unified Learning Framework for Ride  Hailing Platforms

We present a new practical framework based on deep reinforcement learning and
decision-time planning for real-world vehicle repositioning on ride-hailing (a
type of mobility-on-demand, MoD) platforms. Our approach learns the
spatiotemporal state-value function using a batch training algorithm with deep
value networks. The optimal repositioning action is generated on-demand through
value-based policy search, which combines planning and bootstrapping with the
value networks. For the large-fleet problems, we develop several algorithmic
features that we incorporate into our framework and that we demonstrate to
induce coordination among the algorithmically-guided vehicles. We benchmark our
algorithm with baselines in a ride-hailing simulation environment to
demonstrate its superiority in improving income efficiency meausred by
income-per-hour. We have also designed and run a real-world experiment program
with regular drivers on a major ride-hailing platform. We have observed
significantly positive results on key metrics comparing our method with
experienced drivers who performed idle-time repositioning based on their own
expertise.

通过深度强化学习和决策时间规划，为运营商提供实时的出行调度决策，提高车辆调度效率和服务质量。