by mkuthan

mkuthan / example-spark

Spark, Spark Streaming and Spark SQL unit testing strategies

202 Stars 122 Forks Last release: Not found Other 33 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Spark, Spark Streaming and Spark SQL unit testing strategies

Build Status Coverage Status


  • ClockWrapper for efficient clock management in Spark Streaming jobs.
  • Base traits for testing Spark, Spark Streaming and Spark SQL to eliminate boilerplate code.
  • Sample applications to show how to make your code testable.
  • All tests can be run or debugged directly from IDE, or using SBT.
  • All test fixtures are prepared as in-memory data structures.
  • SBT is configured to avoid problems with multiple Spark contexts in the same JVM SPARK-2243.
  • SBT is configured to prepare project assembly for deployment on the cluster.


We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.