Need help with evostrat?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

rasmusbergpalm
158 Stars 9 Forks MIT License 18 Commits 0 Opened issues

Description

A library that makes Evolutionary Strategies (ES) simple to use.

Services available

!
?

Need anything else?

Contributors list

# 14,645
JavaScr...
Shell
MATLAB
Jupyter...
18 commits

EvoStrat

A library that makes Evolutionary Strategies (ES) simple to use.

Installation

pip install evostrat

Usage

pop = PopulationImpl(...) # See complete examples for implementations. 
optim = torch.optim.Adam(pop.parameters()) # Use any torch.optim optimizer
for i in range(N):
    optim.zero_grads()
    pop.fitness_grads(n_samples=200) # Computes approximate gradients
    optim.step()

For complete examples that solves 'LunarLander-v2' see the examples/.

Lunar lander

Description

Evolutionary Strategies is a powerful approach to solve reinforcement learning problems and other optimization problems where the gradients cannot be computed with backprop. See "Evolution strategies as a scalable alternative to reinforcement learning" for an excellent introduction.

In ES the objective is to maximize the expected fitness of a distribution over individuals, referred to as the population. With a few math tricks this objective can be maximized with gradient ascent, even if the fitness function itself is not differentiable.

This library offers

  1. A flexible and natural interface for ES that cleanly separates the environment, the reinforcement learning agent, the population distribution and the optimization.
  2. A plug-and-play approach for reinforcement learning agents with
    torch.nn.Module
    policy networks. See examples/lunar_lander.py and examples/normallunarlander.py.
  3. Several population distributions and variants
    1. Independent Normal. equivalent to OpenAI ES or PEPG depending on whether the standard deviation is fixed or learned. See examples/normallunarlander.py
    2. Multivariate Normal with a full covariance matrix. Similar to CMA-ES. See examples/multivariatenormallunar_lander.py
    3. Categorical. For agents with categorical parameters, demonstrating the ability to handle non-normal distributions. See the examples/binarylunarlander.py.
  4. A simple interface for creating your own populations, without having to derive any gradients! Just subclass Population and implement the sampling process. See the built in populations for inspiration.

Attribution

If you use this software in your academic work please cite

@misc{palm2020,
  author = {Palm, Rasmus Berg},
  title = {EvoStrat},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/rasmusbergpalm/evostrat}}
}

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.