A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
You can install
cargo-cheffrom crates.io with
cargo install cargo-chef
:warning: cargo-chef is not meant to be run locally
Its primary use-case is to speed up container builds by running BEFORE the actual source code is copied over. Don't run it on existing codebases to avoid having files being overwritten.
cargo-chefexposes two commands:
prepareand
cook:
cargo chef --help
cargo-chefUSAGE: cargo chef
SUBCOMMANDS: cook Re-hydrate the minimum project skeleton identified by
cargo chef prepare
and build it to cache dependencies prepare Analyze the current project to determine the minimum subset of files (Cargo.lock and Cargo.toml manifests) required to build it and cache dependencies
prepareexamines your project and builds a recipe that captures the set of information required to build your dependencies.
cargo chef prepare --recipe-path recipe.json
Nothing too mysterious going on here, you can examine the
recipe.jsonfile: it contains the skeleton of your project (e.g. all the
Cargo.tomlfiles with their relative path, the
Cargo.lockfile is available) plus a few additional pieces of information.
Cargo.tomlfiles even if they can be found at the canonical default location (
src/main.rsfor a binary,
src/lib.rsfor a library).
The
recipe.jsonis the equivalent of the Python
requirements.txtfile - it is the only input required for
cargo chef cook, the command that will build out our dependencies:
cargo chef cook --recipe-path recipe.json
If you want to build in
--releasemode:
cargo chef cook --release --recipe-path recipe.json
You can leverage it in a Dockerfile:
FROM lukemathwalker/cargo-chef as planner WORKDIR app COPY . . RUN cargo chef prepare --recipe-path recipe.jsonFROM lukemathwalker/cargo-chef as cacher WORKDIR app COPY --from=planner /app/recipe.json recipe.json RUN cargo chef cook --release --recipe-path recipe.json
FROM rust as builder WORKDIR app COPY . .
Copy over the cached dependencies
COPY --from=cacher /app/target target COPY --from=cacher $CARGO_HOME $CARGO_HOME RUN cargo build --release --bin app
FROM rust as runtime WORKDIR app COPY --from=builder /app/target/release/app /usr/local/bin ENTRYPOINT ["/usr/local/bin/app"]
We are using four stages: the first computes the recipe file, the second caches our dependencies, the third builds the binary and the fourth is our runtime environment.
As long as your dependencies do not change the
recipe.jsonfile will stay the same, therefore the outcome of
cargo cargo chef cook --release --recipe-path recipe.jsonwill be cached, massively speeding up your builds (up to 5x measured on some commercial projects).
If you do not want to use the
lukemathwalker/cargo-chefimage, you can simply install the CLI within the Dockerfile:
FROM rust as planner WORKDIR app # We only pay the installation cost once, # it will be cached from the second build onwards RUN cargo install cargo-chef COPY . . RUN cargo chef prepare --recipe-path recipe.jsonFROM rust as cacher WORKDIR app RUN cargo install cargo-chef COPY --from=planner /app/recipe.json recipe.json RUN cargo chef cook --release --recipe-path recipe.json
FROM rust as builder WORKDIR app COPY . .
Copy over the cached dependencies
COPY --from=cacher /app/target target COPY --from=cacher $CARGO_HOME $CARGO_HOME RUN cargo build --release --bin app
FROM rust as runtime WORKDIR app COPY --from=builder /app/target/release/app /usr/local/bin ENTRYPOINT ["/usr/local/bin/app"]
cargo-chefhas been tested on a few OpenSource projects and some of commercial projects, but our testing has definitely not exhausted the range of possibilities when it comes to
cargo buildcustomisations and we are sure that there are a few rough edges that will have to be smoothed out - please file issues on GitHub.
cargo-chef:
A common alternative is to load a minimal
main.rsinto a container with
Cargo.tomland
Cargo.lockto build a Docker layer that consists of only your dependencies (more info here). This is fragile compared to
cargo-chefwhich will instead:
Dockerfileusing the "manual" approach
cargo cookand
cargo buildmust be executed from the same working directory. If you examine the
*.dfiles under
target/debug/depsfor one of your projects using
catyou will notice that they contain absolute paths referring to the project
targetdirectory. If moved around,
cargowill not leverage them as cached dependencies;
cargo buildwill build local dependencies (outside of the current project) from scratch, even if they are unchanged, due to the reliance of its fingerprinting logic on timestamps (see this long issue on
cargo's repository);
Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.