Need help with sparkmonitor?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

162 Stars 53 Forks Apache License 2.0 214 Commits 30 Opened issues


Monitor Apache Spark from Jupyter Notebook

Services available


Need anything else?

Contributors list

# 204,412
Apache ...
197 commits
# 16,365
Apache ...
1 commit

Build Status

Spark Monitor - An extension for Jupyter Notebook

Note: This project is now maintained at

Google Summer of Code - Final Report

For the google summer of code final report of this project click here


+ =

SparkMonitor is an extension for Jupyter Notebook that enables the live monitoring of Apache Spark Jobs spawned from a notebook. The extension provides several features to monitor and debug a Spark job from within the notebook interface itself.



  • Automatically displays a live monitoring tool below cells that run Spark jobs in a Jupyter notebook
  • A table of jobs and stages with progressbars
  • A timeline which shows jobs, stages, and tasks
  • A graph showing number of active tasks & executor cores vs time
  • A notebook server extension that proxies the Spark UI and displays it in an iframe popup for more details
  • For a detailed list of features see the use case notebooks
  • How it Works

Quick Installation

pip install sparkmonitor
jupyter nbextension install sparkmonitor --py --user --symlink 
jupyter nbextension enable sparkmonitor --py --user            
jupyter serverextension enable --py --user sparkmonitor
ipython profile create && echo "c.InteractiveShellApp.extensions.append('sparkmonitor.kernelextension')" >>  $(ipython profile locate default)/

For more detailed instructions click here

To do a quick test of the extension:

docker run -it -p 8888:8888 krishnanr/sparkmonitor

Integration with ROOT and SWAN

At CERN, the SparkMonitor extension would find two main use cases: * Distributed analysis with ROOT and Apache Spark using the DistROOT module. Here is an example demonstrating this use case. * Integration with SWAN, A service for web based analysis, via a modified container image for SWAN user sessions.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.