Need help with pymarl?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

oxwhirl
610 Stars 172 Forks Apache License 2.0 41 Commits 18 Opened issues

Description

Python Multi-Agent Reinforcement Learning framework

Services available

!
?

Need anything else?

Contributors list

- Please pay attention to the version of SC2 you are using for your experiments. 
- Performance is *not* always comparable between versions. 
- The results in SMAC (https://arxiv.org/abs/1902.04043) use SC2.4.6.2.69232 not SC2.4.10.

Python MARL framework

PyMARL is WhiRL's framework for deep multi-agent reinforcement learning and includes implementations of the following algorithms: - QMIX: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning - COMA: Counterfactual Multi-Agent Policy Gradients - VDN: Value-Decomposition Networks For Cooperative Multi-Agent Learning - IQL: Independent Q-Learning - QTRAN: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning

PyMARL is written in PyTorch and uses SMAC as its environment.

Installation instructions

Build the Dockerfile using

shell
cd docker
bash build.sh

Set up StarCraft II and SMAC:

shell
bash install_sc2.sh

This will download SC2 into the 3rdparty folder and copy the maps necessary to run over.

The requirements.txt file can be used to install the necessary packages into a virtual environment (not recomended).

Run an experiment

python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z

The config files act as defaults for an algorithm or environment.

They are all located in

src/config
.
--config
refers to the config files in
src/config/algs
--env-config
refers to the config files in
src/config/envs

To run experiments using the Docker container:

shell
bash run.sh $GPU python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z

All results will be stored in the

Results
folder.

The previous config files used for the SMAC Beta have the suffix

_beta
.

Saving and loading learnt models

Saving models

You can save the learnt models to disk by setting

save_model = True
, which is set to
False
by default. The frequency of saving models can be adjusted using
save_model_interval
configuration. Models will be saved in the result directory, under the folder called models. The directory corresponding each run will contain models saved throughout the experiment, each within a folder corresponding to the number of timesteps passed since starting the learning process.

Loading models

Learnt models can be loaded using the

checkpoint_path
parameter, after which the learning will proceed from the corresponding timestep.

Watching StarCraft II replays

save_replay
option allows saving replays of models which are loaded using
checkpoint_path
. Once the model is successfully loaded,
test_nepisode
number of episodes are run on the test mode and a .SC2Replay file is saved in the Replay directory of StarCraft II. Please make sure to use the episode runner if you wish to save a replay, i.e.,
runner=episode
. The name of the saved replay file starts with the given
env_args.save_replay_prefix
(map_name if empty), followed by the current timestamp.

The saved replays can be watched by double-clicking on them or using the following command:

python -m pysc2.bin.play --norender --rgb_minimap_size 0 --replay NAME.SC2Replay

Note: Replays cannot be watched using the Linux version of StarCraft II. Please use either the Mac or Windows version of the StarCraft II client.

Documentation/Support

Documentation is a little sparse at the moment (but will improve!). Please raise an issue in this repo, or email Tabish

Citing PyMARL

If you use PyMARL in your research, please cite the SMAC paper.

M. Samvelyan, T. Rashid, C. Schroeder de Witt, G. Farquhar, N. Nardelli, T.G.J. Rudner, C.-M. Hung, P.H.S. Torr, J. Foerster, S. Whiteson. The StarCraft Multi-Agent Challenge, CoRR abs/1902.04043, 2019.

In BibTeX format:

@article{samvelyan19smac,
  title = {{The} {StarCraft} {Multi}-{Agent} {Challenge}},
  author = {Mikayel Samvelyan and Tabish Rashid and Christian Schroeder de Witt and Gregory Farquhar and Nantas Nardelli and Tim G. J. Rudner and Chia-Man Hung and Philiph H. S. Torr and Jakob Foerster and Shimon Whiteson},
  journal = {CoRR},
  volume = {abs/1902.04043},
  year = {2019},
}

License

Code licensed under the Apache License v2.0

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.