One-stop ZFS backup & replication solution
zrepl is a one-stop ZFS backup & replication solution.
User Documentation can be found at zrepl.github.io.
The above does not apply if you already implemented everything. Check out the Coding Workflow section below for details.
This section provides an overview of the zrepl build & release process. Check out
docs/installation/compile-from-source.rstfor build-from-source instructions.
Install build dependencies using
python3-pipto fetch the build dependencies for the docs - you might want to use a venv. If you just want to install the Go dependencies, run
The test suite is split into pure Go tests (
make test-go) and platform tests that interact with ZFS and thus generally require root privileges (
sudo make test-platform). Platform tests run on their own pool with the name
zreplplatformtest, which is created using the file vdev in
For a full code coverage profile, run
make test-go COVER=1 && sudo make test-platform && make cover-merge. An HTML report can be generated using
Code generation is triggered by
make generate. Generated code is committed to the source tree.
Makefileis catering to the needs of developers & CI, not distro packagers. It provides phony targets for * local development (building, running tests, etc) * building a release in Docker (used by the CI & release management) * building .deb and .rpm packages out of the release artifacts.
Build tooling & dependencies are documented as code in
lazy.sh. Go dependencies are then fetched by the go command and pip dependencies are pinned through a
We use CircleCI for continuous integration. There are two workflows:
ciruns for every commit / branch / tag pushed to GitHub. It is supposed to run very fast (<5min and provides quick feedback to developers). It runs formatting checks, lints and tests on the most important OSes / architectures. Artifacts are published to minio.cschwarz.com (see GitHub Commit Status).
masterArtifacts are published to minio.cschwarz.com (see GitHub Commit Status).
Releases are issued via Git tags + GitHub Releases feature. The procedure to issue a release is as follows: * Issue the source release: * Git tag the release on the
masterbranch. * Push the tag. * Run
./docs/publish.shto re-build & push zrepl.github.io. * Issue the official binary release: * Run the
releasepipeline (triggered via CircleCI API) * Download the artifacts to the release manager's machine. * Create a GitHub release, edit the changelog, upload all the release artifacts, including .rpm and .deb files. * Issue the GitHub release. * Add the .rpm and .deb files to the official zrepl repos, publish those.
Official binary releases are not re-built when Go receives an update. If the Go update is critical to zrepl (e.g. a Go security update that affects zrepl), we'd issue a new source release. The rationale for this is that whereas distros provide a mechanism for this (
$zrepl_source_release-$distro_package_revision), GitHub Releases doesn't which means we'd need to update the existing GitHub release's assets, which nobody would notice (no RSS feed updates, etc.). Downstream packagers can read the changelog to determine whether they want to push that minor release into their distro or simply skip it.
sudo make test-platform-bin && sudo make test-platformon a test system to validate that zrepl's abstractions on top of ZFS work with the system ZFS.
hierand logging system.
dist/systemdcontains a Systemd unit template.
./dist, e.g. in
ZREPL_VERSIONvariable and how it passed to Go's
ldFlags. This is how
zrepl versionknows what version number to show. Your build system should set the
ldFlagsflags appropriately and add a prefix or suffix that indicates that the given zrepl binary is a distro build, not an official one.
Backward-incompatible changes must be documented in the git commit message and are listed in
In ZFS, dataset refers to the objects filesystem, ZVOL and snapshot.
However, we need a word for filesystem & ZVOL but not a snapshot, bookmark, etc.
Toward the user, the following terminology is used:
Sadly, the zrepl implementation is inconsistent in its use of these words: variables and types are often named dataset when they in fact refer to a filesystem.
There will not be a big refactoring (an attempt was made, but it's destroying too much history without much gain).
However, new contributions & patches should fix naming without further notice in the commit message.