Need help with kafka-streams?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

bbejeck
223 Stars 98 Forks Apache License 2.0 34 Commits 2 Opened issues

Description

Code examples for working with Kafka Streams

Services available

!
?

Need anything else?

Contributors list

kafka-streams

This is the repository for the examples of using Kafka streams covered in the blog posts:

Requirements to build this project

  1. Java 8
  2. Gradle

Requirements to run the examples

  1. kafka version kafka_2.11-0.10.1.0 see the section marked "Running a task on a particular version of Scala"
  2. The json-data-generator from ACES,Inc

Setup Instructions

Extact the kafka_2.11-0.10.1.0.tgz file

tar -xvzf kafka_2.11-0.10.1.0.tgz

Start zookeeper and kafka

      kafka-install-dir/bin/zookeeper-server-start.sh kafka-install-dir/conf/zookeeper.properties
      kafka-install-dir/bin/kafka-server-start.sh kafka-install-dir/conf/server.properties

Install the Json-Data-Generator

Download the latest json-data-generator release and follow the install instructions here

Setup the kafka-streams repo

Clone or fork the repo

     git clone [email protected]:bbejeck/kafka-streams    
     cd kafka-streams

Then copy the json config files to json generator conf directory
    cp streaming-workflows/* /json-data-generator-1.2.0/conf

Create all the topics required by the examples

     ./bin/create-topics.sh /usr/local/kafka_2.11-0.10.1.0 localhost 2181
     args are kafka home, zookeeper host and zookeeper port adjust accordingly

Running the Purchase Processor API KStreams API Examples

 cd /json-data-generator-1.2.0/
 java -jar json-data-generator-1.2.0 purchases-config.json
 cd kafka-streams
 ./gradlew runPurchaseProcessor | runPurchaseStreams

Running the Stock Trades Processor API or KStreams API Examples

 cd /json-data-generator-1.2.0/
 java -jar json-data-generator-1.2.0 stock-transactions-config.json
 cd kafka-streams
 ./gradlew runStockProcessor | runStockStreams

Running the Twitter KStreams Language Classification Example

rename src/main/resources/twitter-app.properties.template to twitter-app.properties 
fill out the properties file with all the required values

cd kafka-streams ./gradlew runTwitterKstreamNLP

Viewing the results of the purchase streaming examples

cd kafka_install-dir/bin
./kafka-console-consumer --topic [patterns|rewards|purchases] --zookeeper localhost:2181

Viewing the results of the stock-trading streaming examples

cd kafka_install-dir/bin
./kafka-console-consumer --topic [stocks-out|transaction-summary] --zookeeper localhost:2181

Viewing the results of the Twitter KStreams Language Classification Example

cd kafka_install-dir/bin
./kafka-console-consumer --topic [english|french|spanish] --zookeeper localhost:2181    

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.