The SuReLI blogCuriosity Creates Diversity in Policy Search
We are pleased to announce the publication of the research paper titled “Curiosity Creates Diversity in Policy Search” in the journal Transactions on Evolutionary Learning and Optimization. This work was co-authored by Paul-Antoine, Emmanuel, Yann Besse and Dennis.
Abstract of the Research
When searching for policies, reward-sparse environments often lack sufficient information about which behaviors to improve upon or avoid. In such environments, the policy search process is bound to blindly search for reward-yielding transitions and no early reward can bias this search in one direction or another. A way to overcome this is to use intrinsic motivation in order to explore new transitions until a reward is found. In this work, we use a recently proposed definition of intrinsic motivation, Curiosity, in an evolutionary policy search method. We propose Curiosity-ES, an evolutionary strategy adapted to use Curiosity as a fitness metric. We compare Curiosity-ES with other evolutionary algorithms intended for exploration, as well as with Curiosity-based reinforcement learning, and find that Curiosity-ES can generate higher diversity without the need for an explicit diversity criterion and leads to more policies which find reward.
Key Findings
- An algorithm that creates diversity without the need for an explicit diversity criterion.
- Curiosity-ES outperforms other exploration algorithms in sparse-reward scenarios.
- Empirical demonstration that, for some environments, combining the Curiosity exploration bonus with Evolutionary Strategies (ES) maintains a better balance in the inherent two-player game of exploration methods using uncertainty bonuses.
Publication Details
- Title: Curiosity Creates Diversity in Policy Search
- Journal: Transactions on Evolutionary Learning and Optimization
- Link to Publication: Read the full paper here