Need help with zchunk?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

zchunk
157 Stars 15 Forks BSD 2-Clause "Simplified" License 420 Commits 6 Opened issues

Description

A file format designed for highly efficient deltas while maintaining good compression

Services available

!
?

Need anything else?

Contributors list

zchunk

Coverity Scan Build Status
Jenkins Build Status

zchunk is a compressed file format that splits the file into independent chunks. This allows you to only download changed chunks when downloading a new version of the file, and also makes zchunk files efficient over rsync.

zchunk files are protected with strong checksums to verify that the file you downloaded is, in fact, the file you wanted.

As of zchunk-1.0, the ABI and API have been marked stable, and the only changes allowed are backwards-compatible additions

Installation

To build and install zchunk, first install meson and run

meson build
cd build
ninja
ninja test
sudo ninja install

If you're building on an operating system where some libraries are stored in /usr/local, you'll need to replace

meson build
above as follows:
CFLAGS=-I/usr/local/include CXXFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib meson build

Using the utilities

To decompress a zchunk file, simply run:

unzck 

To compress a new zchunk file, run:

zck 

To download a zchunk file, run:

zckdl -s  

To read a zchunk header, run:

zck_read_header 

Zchunk dictionaries

By default, each chunk in a zchunk file is compressed independently. However, if you're creating a zchunk file that has any repetitive data, you may be able to reduce the overall file size by using a zstd dictionary. The dictionary takes up extra space at the beginning of the zchunk file, but is used as an identical initial dictionary for compressing each chunk, which can give a significant overall savings.

It is important that all further revisions of the zchunk file use the same dictionary. If the dictionary changes, none of the chunks will match from the old file, and the full new file will be downloaded.

Zchunk can use any zstd dictionary, but also includes a utility to generate the ideal zstd dictionary for a zchunk file.

To create an ideal dictionary for a zchunk file, run:

zck_gen_zdict 

The dictionary will be saved as

.

You will then need to recompress the file with the dictionary:

zck -D 

Note that

zck_gen_zdict
does require that the
zstd
binary be installed on your system.

Documentation

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.