Need help with bytecount?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

llogiq
152 Stars 19 Forks Other 120 Commits 3 Opened issues

Description

Counting occurrences of a given byte or UTF-8 characters in a slice of memory – fast

Services available

!
?

Need anything else?

Contributors list

# 24,374
Rust
Shell
IPFS
ripgrep
43 commits
# 496,365
Rust
HTML
Shell
11 commits
# 11,329
Rust
no-std
cross-c...
sed
6 commits
# 40,526
Django
google-...
google-...
euler
3 commits
# 5,992
rust-la...
Less
React
ant-des...
2 commits
# 70,656
Shell
Homebre...
agda
quickch...
2 commits
# 135,528
fractal...
Nim
cython
GitHub
2 commits
# 71,210
Raspber...
cross-c...
rust-la...
Groovy
1 commit
# 10,452
Go
Emacs
IPFS
network...
1 commit
# 107,630
GitHub
showcas...
Rust
charset
1 commit

bytecount

Counting bytes really fast

Build Status Windows build status Current Version License: Apache 2.0/MIT

This uses the "hyperscreamingcount" algorithm by Joshua Landau to count bytes faster than anything else. The newlinebench repository has further benchmarks for old versions of this repository.

To use bytecount in your crate, if you have cargo-edit, just type

cargo add bytecount
in a terminal with the crate root as the current path. Otherwise you can manually edit your
Cargo.toml
to add
bytecount = 0.5.1
to your
[dependencies]
section.

In your crate root (

lib.rs
or
main.rs
, depending on if you are writing a library or application), add
extern crate bytecount;
. Now you can simply use
bytecount::count
as follows:
extern crate bytecount;

fn main() { let mytext = "some potentially large text, perhaps read from disk?"; let spaces = bytecount::count(mytext.as_bytes(), b' '); .. }

bytecount supports two features to make use of modern CPU's features to speed up counting considerably. To allow your users to use them, add the following to your

Cargo.toml
:
[features]
runtime-dispatch-simd = ["bytecount/runtime-dispatch-simd"]
generic-simd = ["bytecount/generic-simd"]

The first,

runtime-dispatch-simd
, enables detection of SIMD capabilities at runtime, which allows using the SSE2 and AVX2 codepaths, but cannot be used with
no_std
.

Your users can then compile with runtime dispatch using:

cargo build --release --features runtime-dispatch-simd

The second,

generic-simd
, uses
packed_simd
to provide a fast architecture-agnostic SIMD codepath, but requires running on nightly.

Your users can compile with this codepath using:

cargo build --release --features generic-simd

Building for a more specific architecture will also improve performance. You can do this with

RUSTFLAGS="-C target-cpu=native" cargo build --release

The scalar algorithm is explained in depth here.

Note: Versions until 0.4.0 worked with Rust as of 1.20.0. Version 0.5.0 until 0.6.0 requires Rust 1.26 or later, and at least 1.27.2 to use SIMD. Versions from 0.6.0 require Rust 1.32.0 or later.

License

Licensed under either of at your discretion:

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.