A command-line benchmarking tool
A command-line benchmarking tool.
fdand
find:
To run a benchmark, you can simply call
hyperfine .... The argument(s) can be any shell command. For example:
bash hyperfine 'sleep 0.3'
Hyperfine will automatically determine the number of runs to perform for each command. By default, it will perform at least 10 benchmarking runs. To change this, you can use the
-m/
--min-runsoption:
bash hyperfine --min-runs 5 'sleep 0.2' 'sleep 3.2'
If the program execution time is limited by disk I/O, the benchmarking results can be heavily influenced by disk caches and whether they are cold or warm.
If you want to run the benchmark on a warm cache, you can use the
-w/
--warmupoption to perform a certain number of program executions before the actual benchmark:
bash hyperfine --warmup 3 'grep -R TODO *'
Conversely, if you want to run the benchmark for a cold cache, you can use the
-p/
--prepareoption to run a special command before each timing run. For example, to clear harddisk caches on Linux, you can run
bash sync; echo 3 | sudo tee /proc/sys/vm/drop_cachesTo use this specific command with Hyperfine, call
sudo -vto temporarily gain sudo permissions and then call:
bash hyperfine --prepare 'sync; echo 3 | sudo tee /proc/sys/vm/drop_caches' 'grep -R TODO *'
If you want to run a benchmark where only a single parameter is varied (say, the number of threads), you can use the
-P/
--parameter-scanoption and call:
bash hyperfine --prepare 'make clean' --parameter-scan num_threads 1 12 'make -j {num_threads}'This also works with decimal numbers. The
-D/
--parameter-step-sizeoption can be used to control the step size:
bash hyperfine --parameter-scan delay 0.3 0.7 -D 0.2 'sleep {delay}'This runs
sleep 0.3,
sleep 0.5and
sleep 0.7.
If you are using bash, you can export shell functions to directly benchmark them with hyperfine:
$ my_function() { sleep 1; } $ export -f my_function $ hyperfine my_function
If you are using a different shell, or if you want to benchmark shell aliases, you may try to put them in a separate file:
echo 'my_function() { sleep 1 }' > /tmp/my_function.sh echo 'alias my_alias="sleep 1"' > /tmp/my_alias.sh hyperfine 'source /tmp/my_function.sh; eval my_function' hyperfine 'source /tmp/my_alias.sh; eval my_alias'
Hyperfine has multiple options for exporting benchmark results: CSV, JSON, Markdown (see
--helptext for details). To export results to Markdown, for example, you can use the
--export-markdownoption that will create tables like this:
| Command | Mean [s] | Min [s] | Max [s] | Relative | |:---|---:|---:|---:|---:| |
find . -iregex '.*[0-9]\.jpg$'| 2.275 ± 0.046 | 2.243 | 2.397 | 9.79 ± 0.22 | |
find . -iname '*[0-9].jpg'| 1.427 ± 0.026 | 1.405 | 1.468 | 6.14 ± 0.13 | |
fd -HI '.*[0-9]\.jpg$'| 0.232 ± 0.002 | 0.230 | 0.236 | 1.00 |
The JSON output is useful if you want to analyze the benchmark results in more detail. See the
scripts/folder for some examples.
Download the appropriate
.debpackage from the Release page and install it via
dpkg:
wget https://github.com/sharkdp/hyperfine/releases/download/v1.11.0/hyperfine_1.11.0_amd64.deb sudo dpkg -i hyperfine_1.11.0_amd64.deb
On Fedora, hyperfine can be installed from the official repositories:
dnf install hyperfine
On Alpine Linux, hyperfine can be installed from the official repositories:
apk add hyperfine
On Arch Linux, hyperfine can be installed from the official repositories:
pacman -S hyperfine
On NixOS, hyperfine can be installed from the official repositories:
nix-env -i hyperfine
Hyperfine can be installed via xbps
xbps-install -S hyperfine
Hyperfine can be installed via Homebrew:
brew install hyperfine
Or you can install using MacPorts:
sudo port selfupdate sudo port install hyperfine
Hyperfine can be installed via pkg:
pkg install hyperfine
doas pkg_add hyperfine
Hyperfine can be installed via
condafrom the
conda-forgechannel:
conda install -c conda-forge hyperfine
Hyperfine can be installed via cargo:
cargo install hyperfine
Make sure that you use Rust 1.43 or higher.
Download the corresponding archive from the Release page.
Hyperfine is inspired by bench.
Chronologer is a tool that uses
hyperfineto visualize changes in benchmark timings across your Git history.
scriptsfolder in this repository for a set of tools to work with
hyperfinebenchmark results.
The name hyperfine was chosen in reference to the hyperfine levels of caesium 133 which play a crucial role in the definition of our base unit of time — the second.
Copyright (c) 2018-2020 The hyperfine developers
hyperfineis distributed under the terms of both the MIT License and the Apache License 2.0.
See the LICENSE-APACHE and LICENSE-MIT files for license details.