Credit Assignment and Focused Exploration for Sparse-reward Multi-agent Deep Reinforcement Learning

Shuai Han, Mehdi Dastani, Shihan Wang; 7, 2026.

Presented at the Reinforcement Learning Conference (RLC), Montréal, Quebec, Canada, August 15–17, 2026.

Abstract

Recent advances in multi-agent deep reinforcement learning (MADRL) have achieved strong performance in various scenarios. However, training cooperative policies in sparse-reward scenarios remains a major challenge for MADRL due to the unfocused exploration and ambiguous credit assignment. In this paper, we introduce Influence Scope of Agents (ISA), an algorithm that leverages agents' influence for efficient policy training. By evaluating the mutual dependence between agents' actions and states, it automatically learns the scope of state dimensions (attributes) that can be influenced by individual agents. These influence scopes are then used to focus agents' exploration on their controllable aspects of the environment and to calculate credit assignment among agents according to their influence. We evaluate ISA in a variety of sparse-reward multi-agent scenarios. The results show that our method significantly outperforms state-of-the-art baselines.

[abs][pdf]

BibTeX

@article{han2026credit,
    title={Credit Assignment and Focused Exploration for Sparse-reward Multi-agent Deep Reinforcement Learning},
    author={Shuai Han and Mehdi Dastani and Shihan Wang},
    journal={Reinforcement Learning Journal},
    volume={7},
    pages={},
    year={2026}
}