by MarcusOlivecrona

MarcusOlivecrona /REINVENT

Molecular De Novo design using Recurrent Neural Networks and Reinforcement Learning

149 Stars 69 Forks Last release: over 3 years ago (v1.0.1) MIT License 39 Commits 2 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:


Molecular De Novo design using Recurrent Neural Networks and Reinforcement Learning

Searching chemical space as described in:

Molecular De Novo Design through Deep Reinforcement Learning

Video demonstrating an Agent trained to generate analogues to Celecoxib


The current version is a PyTorch implementation that differs in several ways from the original implementation described in the paper. This version works better in most situations and is better documented, but for the purpose of reproducing results from the paper refer to Release v1.0.1

Differences from implmentation in the paper: * Written in PyTorch/Python3.6 rather than TF/Python2.7 * SMILES are encoded with token index rather than as a onehot of the index. An embedding matrix is then used to transform the token index to a feature vector. * Scores are in the range (0,1). * A regularizer that penalizes high values of total episodic likelihood is included. * Sequences are only considered once, ie if the same sequence is generated twice in a batch only the first instance contributes to the loss. * These changes makes the algorithm more robust towards local minima, means much higher values of sigma can be used if needed.


This package requires: * Python 3.6 * PyTorch 0.1.12 * RDkit * Scikit-Learn (for QSAR scoring function) * tqdm (for training Prior)


To train a Prior starting with a SMILES file called mols.smi:

  • First filter the SMILES and construct a vocabulary from the remaining sequences.

    ./ mols.smi
    - Will generate data/mols_filtered.smi and data/Voc. A filtered file containing around 1.1 million SMILES and the corresponding Voc is contained in "data".
  • Then use

    to train the Prior. A pretrained Prior is included.

To train an Agent using our Prior, use the script. For example:

  • ./ --scoring-function activity_model --num-steps 1000

Training can be visualized using the Vizard bokeh app. The is used to log information (by default to data/logs) such as structures generated, average score, and network weights.

  • cd Vizard
  • ./ ../data/logs
  • Open the browser at http://localhost:5006/Vizard

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.