a distributed docker image storage
Speedy is a high performance distributed docker image storage solution written by go/c. It can be easily scaled out by adding more storage instance and no data have to move around between storage instance.
Docker-registry backend storage driver for speedy, It divides docker image layer into fixed-size chunk and uploads/downloads concurrently .
It is a stateless frond-end proxy server designed to provide restful api to upload and download docker image. imageserver get chunkserver information and file id from chunkmaster periodically. imageserver choose a suitable chunkserver group to storage docker image according to chunkserver information independently, we can start many imageserver to provice service at the same time and docker-registry-speedy-driver can use anyone of them equally.
It is a central master node designed to maintain chunkserver information and allocate the file id. chunkmaster store chunkserver information to mysql and keep in memory as cache, while imageserver try to get chunkserver information chunkmaster send the information in memory to imageserver. while imageserver try to get file id, chunkmaster allocate a continuous range of file id and send to imageserver.
It is a highly optimized storage engine for performance and space efficiency.It appends single small image file into large files and maintain file index in memory keeping the IO overhead to a minimum. Normally, a chunkserver group is consist of 3 chunkservers, imageserver writes data to a chunkserver group suceess means storing data to each chunkserver of the group success.
It is an another distributed key-value storage, since It's not open-source yet, you can use mysql instead which store the image layer metadata informations.
After that you can push and pull docker images.
We made a performance test about upload and download of Speedy. We use 4 normal servers
(CPU: 24 core 2G HZ; Memory: 16G; Disk: 300G 10k SAS; Ethernet: 1 Gigabit) to construct
our test environment.
imageserver [node1] / | \ / | \ chunk-server[node2] chunk-server[node3] chunk-server [node4]
We simply use mysql instead of our internal MetaServer, at the same time,
chunkserver is on the default buffer io model.
|concurrent||fileSize(M)||file count||upload time(seconds)||download time(seconds)||upload speed(M/s)||download speed(M/s)|
We can easily got that download speed reach the limit of Ethernet, about 110 M/s.
Although upload speed looks like just 1/3 of download speed, acctually upload also
reach the Ethernet limit, that is the result of upload will concurrently write three