Need help with sltbench?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

ivafanas
139 Stars 6 Forks Boost Software License 1.0 267 Commits 11 Opened issues

Description

C++ benchmark tool. Practical, stable and fast performance testing framework.

Services available

!
?

Need anything else?

Contributors list

# 356,941
cpluspl...
cpp-lib...
benchma...
C++
223 commits
# 373,849
cheat
Shell
cpluspl...
cpp-lib...
8 commits
# 450,500
cpluspl...
cpp-lib...
benchma...
benchma...
1 commit
# 76,754
Nim
zig
benchma...
test-fr...
1 commit
# 729,049
automat...
Shell
PHP
eigen
1 commit

sltbench

Build Status Build status

C++ benchmark tool. Practical, stable and fast performance testing framework. - practical - came up from daily production tasks. Usability matters - stable - correctness and reproducibility is a goal - fast - if results are stable, they should be produced as soon as possible

This tool is trained to gain about 5% measure stability (depends on function, input and context). Of course, it cannot strongly guarantee 5% stability (some functions are unstable itself, nothing helps), but it tends to.

Why sltbench

The closest and the most famous analogue is googlebench - microbenchmarking library from Google engineers.

Key features of

sltbench
are: * sltbench is about 4.7x times faster than googlebench (test execution time). * sltbench produces results with less timing error, which matters for performance results reproducibility. About 5x times improvement for testing dataset. * Supports correct mutable fixtures. * Supports lazy arguments generation including reading from file, autogeneration etc.. (critical for large input datasets which does not fit into RAM all together) * Output customization, separate tools for performance results comparison and more.

measure time

(Results below are obtained on 4 cores i7-2600 CPU @ 3.40GHz, 8Gb RAM for sltbench dataset; see "Metrics" section for details; see How to benchmark section for instructions)

| |sltbench|googlebench|nonius| |----------------------|:------:|:---------:|:----:| | benchmark time, sec | 87.45 | 416.44 |482.03| | average rel error, % | 0.4% | 5.6% | 2.6%| | maximum rel error, % | 2.9% | 15.1% | 60% |

4.7x times speedup might be useful for projects with big performance tests count. For the original project regression performance testing tooks about a week and testing time reduction matters.

Less relative error means: * Less false speedups/slowdowns you get during testing. Economy of developer time. * More precise performance changes may be obtained. If testing tool provides 15% relative error than 10% speedup couldn't be measured.

compilation time

Sltbench contains improvements on compilation time of benchmarks (not library!) for the most popular api since 2.4.0. Benchmarks for functions without arguments and driver launch might be implemented in a more lightweight manner. See Advanced user guide for explanation how to reuse improvements.

(benchcompile metric results below are obtained on Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz, clang-6.0.0, dataset=simple for optimized cases only)

| | compilation time, sec | |--------------------|:---------------------:| | sltbench 2.4.0 | 2.5 | | googlebench v1.5.1 | 16.5 |

Motivating example

The following code registers

my_function
for timing. Just compile it (do not forget about includes) and the executable file will test
my_function
for performance.
void my_function()
{
    std::vector v(100000, 0);
    std::sort(v.begin(), v.end());
}

SLTBENCH_FUNCTION(my_function);

SLTBENCH_MAIN();

Documentation

License

sltbench is released under the Apache 2.0 license

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.