Need help with reinforcement-learning-car?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

452 Stars 204 Forks MIT License 170 Commits 10 Opened issues


Using reinforcement learning to teach a car to avoid obstacles.

Services available


Need anything else?

Contributors list

# 30,802
Deep le...
136 commits
# 587,523
1 commit
# 570,156
1 commit

Using reinforcement learning to train an autonomous vehicle to avoid obstacles

NOTE: If you're coming here from parts 1 or 2 of the Medium posts, you want to visit the releases section and check out version 1.0.0, as the code has evolved passed that.

This is a hobby project I created to learn the basics of reinforcement learning. It uses Python3, Pygame, Pymunk, Keras and Theanos. It employes a Q-learning (unsupervised) algorithm to learn how to move an object around a screen (drive itself) without running into obstacles.

The purpose of this project is to eventually use the learnings from the game to operate a real-life remote-control car, using distance sensors. I am carrying on that project in another GitHub repo here:

This version of the code attempts to simulate the use of sensors to get us a step closer to being able to use this in the real world.

Full writeups that pertain to version 1.0.0 can be found here:

Part 1:

Part 2:

Part 3 (for this version of the code):


These instructions are for a fresh Ubuntu 16.04 box. Most of the same should apply to OS X. If you have issues installing, feel free to open an issue with your error and I'll do my best to help.


Recent Ubuntu releases come with python3 installed. I use pip3 for installing dependencies so install that with

sudo apt install python3-pip
. Install git if you don't already have it with
sudo apt install git

Then clone this repo with

git clone
. It has some pretty big weights files saved in past commits, so to just get the latest the fastest, do
git clone --depth 1

Python dependencies

pip3 install numpy keras h5py

That should install a slew of other libraries you need as well.

Install Pygame

Install Pygame's dependencies with:

sudo apt install mercurial libfreetype6-dev libsdl-dev libsdl-image1.2-dev libsdl-ttf2.0-dev libsmpeg-dev libportmidi-dev libavformat-dev libsdl-mixer1.2-dev libswscale-dev libjpeg-dev

Then install Pygame itself:

pip3 install hg+

Install Pymunk

This is the physics engine used by the simulation. It just went through a pretty significant rewrite (v5) so you need to grab the older v4 version. v4 is written for Python 2 so there are a couple extra steps.

Go back to your home or downloads and get Pymunk 4:


Unpack it:

tar zxvf pymunk-4.0.0.tar.gz

Update from Python 2 to 3:

cd pymunk-pymukn-4.0.0/pymunk

2to3 -w *.py

Install it:

cd ..
python3 install

Now go back to where you cloned

and make sure everything worked with a quick
. If you see a screen come up with a little dot flying around the screen, you're ready to go!


First, you need to train a model. This will save weights to the

folder. You may need to create this folder before running. You can train the model by running:


It can take anywhere from an hour to 36 hours to train a model, depending on the complexity of the network and the size of your sample. However, it will spit out weights every 25,000 frames, so you can move on to the next step in much less time.


Edit the
file to change the path name for the model you want to load. Sorry about this, I know it should be a command line argument.

Then, watch the car drive itself around the obstacles!


That's all there is to it.


Once you have a bunch of CSV files created via the learning, you can convert those into graphs by running:


This will also spit out a bunch of loss and distance averages at the different parameters.


I'm grateful to the following people and the work they did that helped me learn how to do this:

  • Playing Atari with Deep Reinforcement Learning -
  • Deep learning to play Atari games:
  • Another deep learning project for video games:
  • A great tutorial on reinforcement learning that a lot of my project is based on:

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.