We study a sequential profit-maximization problem, optimizing for both price
and ancillary variables like marketing expenditures. Specifically, we aim to
maximize profit over an arbitrary sequence of multiple demand curves, each
dependent on a distinct ancillary variable, but sharing the same price. A
prototypical example is targeted marketing, where a firm (seller) wishes to
sell a product over multiple markets. The firm may invest different marketing
expenditures for different markets to optimize customer acquisition, but must
maintain the same price across all markets. Moreover, markets may have
heterogeneous demand curves, each responding to prices and marketing
expenditures differently. The firm's objective is to maximize its gross profit,
the total revenue minus marketing costs.
Our results are near-optimal algorithms for this class of problems in an
adversarial bandit setting, where demand curves are arbitrary non-adaptive
sequences, and the firm observes only noisy evaluations of chosen points on the
demand curves. We prove a regret upper bound of
$\widetilde{\mathcal{O}}\big(nT^{3/4}\big)$ and a lower bound of
$\Omega\big((nT)^{3/4}\big)$ for monotonic demand curves, and a regret bound of
$\widetilde{\Theta}\big(nT^{2/3}\big)$ for demands curves that are monotonic in
price and concave in the ancillary variables.

该研究针对具有不同需求曲线的顺序盈利最大化问题，考虑价格和市场支出等附属变量的优化，提出了接近最佳的算法，并给出了不同类型需求曲线的后悔上限。

针对性营销的赌徒式利润最大化

Bandit Profit-maximization for Targeted Marketing

Neural replicator dynamics (NeuRD) is an alternative to the foundational
softmax policy gradient (SPG) algorithm motivated by online learning and
evolutionary game theory. The NeuRD expected update is designed to be nearly
identical to that of SPG, however, we show that the Monte Carlo updates differ
in a substantial way: the importance correction accounting for a sampled action
is nullified in the SPG update, but not in the NeuRD update. Naturally, this
causes the NeuRD update to have higher variance than its SPG counterpart.
Building on implicit exploration algorithms in the adversarial bandit setting,
we introduce capped implicit exploration (CIX) estimates that allow us to
construct NeuRD-CIX, which interpolates between this aspect of NeuRD and SPG.
We show how CIX estimates can be used in a black-box reduction to construct
bandit algorithms with regret bounds that hold with high probability and the
benefits this entails for NeuRD-CIX in sequential decision-making settings. Our
analysis reveals a bias--variance tradeoff between SPG and NeuRD, and shows how
theory predicts that NeuRD-CIX will perform well more consistently than NeuRD
while retaining NeuRD's advantages over SPG in non-stationary environments.

论文介绍了一种名为神经复制动力学（NeuRD）的算法，通过引入隐式探索算法来构建 NeuRD-CIX 算法，并在序贯决策制定中展示了它的性能。