Need help with example-spark?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

mkuthan
204 Stars 126 Forks Other 33 Commits 2 Opened issues

Description

Spark, Spark Streaming and Spark SQL unit testing strategies

Services available

!
?

Need anything else?

Contributors list

# 288,112
Scala
Google ...
beam
ml
1 commit

Spark, Spark Streaming and Spark SQL unit testing strategies

Build Status Coverage Status

Features

  • ClockWrapper for efficient clock management in Spark Streaming jobs.
  • Base traits for testing Spark, Spark Streaming and Spark SQL to eliminate boilerplate code.
  • Sample applications to show how to make your code testable.
  • All tests can be run or debugged directly from IDE, or using SBT.
  • All test fixtures are prepared as in-memory data structures.
  • SBT is configured to avoid problems with multiple Spark contexts in the same JVM SPARK-2243.
  • SBT is configured to prepare project assembly for deployment on the cluster.

References

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.