Rust client for Apache Kafka
This project is starting to be maintained by John Ward, the current status is that I am bringing the project up to date with the latest dependencies, removing deprecated Rust code and adjusting the tests.
Welcome to kafka-rust's new home: https://github.com/kafka-rust
This crate works with Cargo and is on crates.io. The API is currently under heavy movement although we do follow semantic versioning (but expect the version number to grow quickly).
[dependencies] kafka = "0.8"
To build kafka-rust the usual
cargo buildshould suffice. The crate supports various features which can be turned off at compile time. See kafka-rust's
Cargo.tomland cargo's documentation.
kafka-rustis tested for compatibility with Kafka 0.8.2 and newer. However, not all features from Kafka 0.9 and newer are supported yet.
This is a higher-level consumer API for Kafka and is provided by the module
kafka::consumer. It provides convenient offset management support on behalf of a specified group. This is the API a client application of this library wants to use for receiving messages from Kafka.
This is a higher-level producer API for Kafka and is provided by the module
kafka::producer. It provides convenient automatic partition assignment capabilities through partitioners. This is the API a client application of this library wants to use for sending messsages to Kafka.
kafka::clientmodule is the central point of this API. However, this is a mid-level abstraction for Kafka rather suitable for building higher-level APIs. Applications typically want to use the already mentioned
Producer. Nevertheless, the main features or
There's still a lot of room for improvement on
kafka-rust. Not everything works right at the moment, and testing coverage could be better. Use it in production at your own risk. Have a look at the issue tracker and feel free to contribute by reporting new problems or contributing to existing ones. Any constructive feedback is warmly welcome!
As usually with open source, don't hesitate to fork the repo and submit a pull request if you see something to be changed. We'll be happy to see
kafka-rustimproving over time.
When working locally, the integration tests require that you must have Docker (1.10.0+) and docker-compose (1.6.0+) installed and run the tests via the included
run-all-testsscript in the
testsdirectory. See the
run-all-testsscript itself for details on its usage.
Note unless otherwise explicitly stated in the documentation, this library will ignore requests to topics which it doesn't know about. In particular it will not try to retrieve messages from non-existing/unknown topics. (This behavior is very likely to change in future version of this library.)
Given a local kafka server installation you can create topics with the following command (where
kafka-topics.shis part of the Kafka distribution):
kafka-topics.sh --topic my-topic --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1
See also Kafka's quickstart guide for more information.
librdkafka. rust-rdkafka provides a safe Rust interface to librdkafka.