The effectiveness of advertising in e-commerce largely depends on the ability
of merchants to bid on and win impressions for their targeted users. The
bidding procedure is highly complex due to various factors such as market
competition, user behavior, and the diverse objectives of advertisers. In this
paper we consider the problem at the level of user timelines instead of
individual bid requests, manipulating full policies (i.e. pre-defined bidding
strategies) and not bid values. In order to optimally allocate policies to
users, typical multiple treatments allocation methods solve knapsack-like
problems which aim at maximizing an expected value under constraints. In the
industrial contexts such as online advertising, we argue that optimizing for
the probability of success is a more suited objective than expected value
maximization, and we introduce the SuccessProbaMax algorithm that aims at
finding the policy allocation which is the most likely to outperform a fixed
reference policy. Finally, we conduct comprehensive experiments both on
synthetic and real-world data to evaluate its performance. The results
demonstrate that our proposed algorithm outperforms conventional expected-value
maximization algorithms in terms of success rate.

电子商务广告效果主要取决于商家对目标用户进行竞标和赢得展示的能力。本文考虑在用户时间线级别上解决这一问题，通过操纵完整策略而不是竞标值来将策略最优地分配给用户，引入 SuccessProbaMax 算法来优化成功概率，实验证明该算法在成功率方面优于传统预期值最大化算法。

在在线系统中最大化策略分配的成功概率

Maximizing the Success Probability of Policy Allocations in Online  Systems

Real-time advertising allows advertisers to bid for each impression for a
visiting user. To optimize specific goals such as maximizing revenue and return
on investment (ROI) led by ad placements, advertisers not only need to estimate
the relevance between the ads and user's interests, but most importantly
require a strategic response with respect to other advertisers bidding in the
market. In this paper, we formulate bidding optimization with multi-agent
reinforcement learning. To deal with a large number of advertisers, we propose
a clustering method and assign each cluster with a strategic bidding agent. A
practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed
and implemented to balance the tradeoff between the competition and cooperation
among advertisers. The empirical study on our industry-scaled real-world data
has demonstrated the effectiveness of our methods. Our results show
cluster-based bidding would largely outperform single-agent and bandit
approaches, and the coordinated bidding achieves better overall objectives than
purely self-interested bidding agents.

本文讨论了如何利用多智能体强化学习进行实时广告优化，提出了一种聚类算法分配策略代理，并通过行业实验表明，与单一代理和实验算法相比，基于聚类的竞价模型具有更好的效果。

基于多智能 Agent 强化学习的实时竞价展示广告

Real-Time Bidding with Multi-Agent Reinforcement Learning in Display  Advertising

Bridge is among the zero-sum games for which artificial intelligence has not
yet outperformed expert human players. The main difficulty lies in the bidding
phase of bridge, which requires cooperative decision making under partial
information. Existing artificial intelligence systems for bridge bidding rely
on and are thus restricted by human-designed bidding systems or features. In
this work, we propose a pioneering bridge bidding system without the aid of
human domain knowledge. The system is based on a novel deep reinforcement
learning model, which extracts sophisticated features and learns to bid
automatically based on raw card data. The model includes an
upper-confidence-bound algorithm and additional techniques to achieve a balance
between exploration and exploitation. Our experiments validate the promising
performance of our proposed model. In particular, the model advances from
having no knowledge about bidding to achieving superior performance when
compared with a champion-winning computer bridge program that implements a
human-designed bidding system.

本研究提出了一种新型的深度强化学习模型，不依赖于人类领域知识，基于原始卡牌数据自动学习出复杂的特征并完成桥牌叫牌，实验证明该模型在与实现人类设计叫牌系统的计算机桥牌程序进行对比时表现出了优异的性能。

基于深度强化学习的自动桥牌叫牌

Automatic Bridge Bidding Using Deep Reinforcement Learning

A Richman game is a combinatorial game in which, rather than alternating
moves, the two players bid for the privilege of making the next move. We
consider both the case where the players pay each other and the case where the
players pay a neutral third party. We find optimal strategies considering both
the case where the players know how much money their opponent has and the case
where they do not.

本文研究了 Richman game 这种组合游戏中的最优策略，包括双方互相付款和与中立第三方付款的情况，考虑了双方知道对方有多少钱和不知道的情况。