A library that makes Evolutionary Strategies (ES) simple to use.
A library that makes Evolutionary Strategies (ES) simple to use.
pip install evostrat
pop = PopulationImpl(...) # See complete examples for implementations. optim = torch.optim.Adam(pop.parameters()) # Use any torch.optim optimizer for i in range(N): optim.zero_grads() pop.fitness_grads(n_samples=200) # Computes approximate gradients optim.step()
For complete examples that solves 'LunarLander-v2' see the examples/.
Evolutionary Strategies is a powerful approach to solve reinforcement learning problems and other optimization problems where the gradients cannot be computed with backprop. See "Evolution strategies as a scalable alternative to reinforcement learning" for an excellent introduction.
In ES the objective is to maximize the expected fitness of a distribution over individuals, referred to as the population. With a few math tricks this objective can be maximized with gradient ascent, even if the fitness function itself is not differentiable.
This library offers
torch.nn.Modulepolicy networks. See examples/lunar_lander.py and examples/normallunarlander.py.
If you use this software in your academic work please cite
@misc{palm2020, author = {Palm, Rasmus Berg}, title = {EvoStrat}, year = {2020}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/rasmusbergpalm/evostrat}} }