Communication Dans Un Congrès Année : 2024

Strategic Multi-Armed Bandit Problems Under Debt-Free Reporting

Résumé

We consider the classical multi-armed bandit problem, but with strategic arms. In this context, each arm is characterized by a bounded support reward distribution and strategically aims to maximize its own utility by potentially retaining a portion of its reward, and disclosing only a fraction of it to the learning agent. This scenario unfolds as a game over T rounds, leading to a competition of objectives between the learning agent, aiming to minimize their regret, and the arms, motivated by the desire to maximize their individual utilities. To address these dynamics, we introduce a new mechanism that establishes an equilibrium wherein each arm behaves truthfully and discloses as much of its rewards as possible. With this mechanism, the agent can attain the second-highest average (true) reward among arms, with a cumulative regret bounded by O(log(T )/∆) (problem-dependent) or O( T log(T )) (worst-case).

This perspective leaves out an array of dynamic agency dilemmas, where the player selects one of the K agents (arms) at each time step to execute a task on his behalf, with the associated cost remaining hidden from the player due to his limited domain or market knowledge. Essentially, the player is uncertain about the precise costs or returns until the task is completed, and the agent has substantial freedom to set these ex-post [8]. In this context, it is reasonable to assume that arms are strategic and may act in their self-interest. Conceptually, arms can report a value, denoted as x kt,t , which may differ from the actual reward r kt,t , retaining in the process a net utility of r kt,t -x kt,t . This strategic scenario introduces a game-like dynamic, creating a competition of objectives between the player, aiming to minimize his regret and the arms, wishing to maximize their own utilities.

Motivated by numerous real-world applications where, by design, strategic arms operate under restricted payment conditions, our study focuses on debt-free reporting. In this setting, arms cannot report values higher than the observed reward. An illustrative example pertains to scenarios with binary variables, where declaring a fake failure is possible but creating a fake success is impossible. This situation is evident in e-commerce, where advertisers receive rewards for successful ad campaigns that result in a sale after a click. Notably, an e-commerce platform may choose to conceal a sale, but it cannot fabricate one. This setting is also referred to as budget-balance in the repeated trade literature [9, 10], requiring x kt,t ≤ r kt,t for all t, or equivalently, (x kt,t , r kt,t ) belonging to the upper triangle T = {(a, b) ∈ S 2 | a ≤ b}, where S represents the variables' space.

Fichier principal
Vignette du fichier
3205_Strategic_Multi_Armed_Ban (1).pdf (790.79 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04912557 , version 1 (26-01-2025)

Identifiants

  • HAL Id : hal-04912557 , version 1

Citer

Ahmed Ben Yahmed, Clément Calauzènes, Vianney Perchet. Strategic Multi-Armed Bandit Problems Under Debt-Free Reporting. The Thirty-Eighth Annual Conference on Neural Information Processing Systems, Dec 2024, Vancouver, France. ⟨hal-04912557⟩
0 Consultations
0 Téléchargements

Partager

More