Need help with jocko?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

travisjeffery
4.4K Stars 314 Forks MIT License 532 Commits 58 Opened issues

Description

Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

Services available

!
?

Need anything else?

Contributors list

Jocko

ci gitter codecov

Kafka/distributed commit log service in Go.

Goals of this project:

  • Implement Kafka in Go
  • Protocol compatible with Kafka so Kafka clients and services work with Jocko
  • Make operating simpler
  • Distribute a single binary
  • Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper)
  • Smarter configuration settings
    • Able to use percentages of disk space for retention policies rather than only bytes and time kept
    • Handling size configs when you change the number of partitions or add topics
  • Learn a lot and have fun

TODO

  • [x] Producing
  • [x] Fetching
  • [x] Partition consensus and distribution
  • [ ] Protocol
    • [x] Produce
    • [x] Fetch
    • [x] Metadata
    • [x] Create Topics
    • [x] Delete Topics
    • [ ] Consumer group [current task]
  • [x] Discovery
  • [ ] API versioning [more API versions to implement]
  • [ ] Replication [first draft done - testing heavily now]

Hiatus Writing Book

I’m writing a book for PragProg called Building Distributed Services with Go. You can sign up on this mailing list and get updated when the book’s available. It walks you through building a distributed commit log from scratch. I hope it will help Jocko contributors and people who want to work on distributed services.

Reading

Project Layout

├── broker        broker subsystem
├── cmd           commands
│   └── jocko     command to run a Jocko broker and manage topics
├── commitlog     low-level commit log implementation
├── examples      examples running/using Jocko
│   ├── cluster   example booting up a 3-broker Jocko cluster
│   └── sarama    example producing/consuming with Sarama
├── protocol      golang implementation of Kafka's protocol
├── prometheus    wrapper around Prometheus' client lib to handle metrics
├── server        API subsystem
└── testutil      test utils
    └── mock      mocks of the various subsystems

Building

Local

  1. Clone Jocko

    $ go get github.com/travisjeffery/jocko
    
  2. Build Jocko

    $ cd $GOPATH/src/github.com/travisjeffery/jocko
    $ make build
    

    (If you see an error about

    dep
    not being found, ensure that
    $GOPATH/bin
    is in your
    PATH
    )

Docker

docker build -t travisjeffery/jocko:latest .

Contributing

See CONTRIBUTING for details on submitting patches and the contribution workflow.

License

Jocko is under the MIT license, see the LICENSE file for details.


We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.