INACTIVE: Docker garbage collection of containers and images
We're not developing or accepting new features.
A simple Docker container and image garbage collection script.
Although docker normally prevents removal of images that are in use by containers, we take extra care to not remove any image tags (e.g., ubuntu:14.04, busybox, etc) that are in use by containers. A naive
docker rmi $(docker images -q)will leave images stripped of all tags, forcing docker to re-pull the repositories when starting new containers even though the images themselves are still on disk.
This script is intended to be run as a cron job, but you can also run it as a Docker container (see below).
sudo apt-get install git devscripts debhelper build-essential dh-make git clone https://github.com/spotify/docker-gc.git cd docker-gc debuild -us -uc -b
If you get lintian errors during
debuild --no-lintian -us -uc -b.
sudo dpkg -i ../docker-gc_0.1.0_all.deb
This installs the
/usr/sbin. If you want it to run as a cron job, you can configure it now by creating a root-owned executable file
/etc/cron.hourly/docker-gcwith the following contents:
To test that the job will actually run you can use this command
run-parts --test /etc/cron.hourly
To use the script manually, run
docker-gc. The system user under which
docker-gcruns needs to have read and write access to the
$STATE_DIRenvironment variable which defaults to
There can be images that are large that serve as a common base for many application containers, and as such, make sense to pin to the machine, as many derivative containers will use it. This can save time in pulling those kinds of images. There may be other reasons to exclude images from garbage collection. To do so, create
/etc/docker-gc-exclude, or if you want the file to be read from elsewhere, set the
EXCLUDE_FROM_GCenvironment variable to its location. This file can contain image name patterns (in the
grepsense), one per line, such as
spotify/cassandra:latestor it can contain image ids (truncated to the length shown in
docker imageswhich is 12.
An example image excludes file might contain:
spotify/cassandra:latest redis:.* 9681260c3ad5
There can also be containers (for example data only containers) which you would like to exclude from garbage collection. To do so, create
/etc/docker-gc-exclude-containers, or if you want the file to be read from elsewhere, set the
EXCLUDE_CONTAINERS_FROM_GCenvironment variable to its location. This file should contain name patterns (in the
grepsense), one per line, such as
An example container excludes file might contain:
There can be occasions where you don't want to remove a dangling volume. To enable this functionality you can create a file named
EXCLUDE_VOLUMES_IDS_FILEenv var with any path for such file), containing name patterns (in the
grepsense), one per line, of volumes that will be excluded from garbage collection.
By default, docker will not remove an image if it is tagged in multiple repositories. If you have a server running docker where this is the case, for example in CI environments where dockers are being built, re-tagged, and pushed, you can enable a force flag to override this default.
You might want to always keep a set of the most recent images for any repository. For example, if you are continually rebuilding an image during development you would want to clear out all but the most recent version of an image. To do so, set the
MINIMUM_IMAGES_TO_SAVE=1environment variable. You can preserve any count of the most recent images, e.g. save the most recent 10 with
By default, if an error is encountered when cleaning up a container, Docker will report the error back and leave it on disk. This can sometimes lead to containers accumulating. If you run into this issue, you can force the removal of the container by setting the environment variable below:
By default, docker-gc will not remove a container if it exited less than 3600 seconds (1 hour) ago. In some cases you might need to change this setting (e.g. you need exited containers to stick around for debugging for several days). Set the
GRACE_PERIOD_SECONDSvariable to override this default.
This setting also prevents the removal of images that have been created less than
By default, docker-gc will proceed with deletion of containers and images. To test your command-line options set the
DRY_RUNvariable to override this default.
A Dockerfile is provided as an alternative to a local installation. By default the container will start up, run a single garbage collection, and shut down.
The image is published as
The image is currently built with Docker 17.09.0-ce, but to build it against a newer Docker version (to ensure that the API version of the command-line interface matches with your Docker daemon), simply edit theline in
Dockerfileprior to the build step below.
Build the Docker image with
make -f Makefile.docker imageor:
docker build -t spotify/docker-gc .
The docker-gc container requires access to the docker socket in order to function, so you need to map it when running, e.g.:
docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc:ro spotify/docker-gc
/etcdirectory is also mapped so that it can read any exclude files that you've created.
If you want to remove volumes, you can do so by passing REMOVE_VOLUMES env var set to 1.
$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc -e REMOVE_VOLUMES=1 spotify/docker-gc
If you want to remove volumes only for a specified driver, you can do it by passing VOLUMEDELETEONLY_DRIVER env var set to the driver name.
If your docker daemon is configured to run with user namespace, you will need to run the container with user namespace disabled:
docker run --rm --userns host -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc spotify/docker-gc