Need help with LearningApacheSpark?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

170 Stars 77 Forks MIT License 36 Commits 0 Opened issues



Services available


Need anything else?

Contributors list

# 188,695
Apache ...
16 commits

Learning Apache Spark


This is a shared repository for Learning Apache Spark Notes. The first version was posted on Github in [Feng2017]. This shared repository mainly contains the self-learning and self-teaching notes from Wenqiang during his IMA Data Science Fellowship.

In this repository, I try to use the detailed demo code and examples to show how to use each main functions. If you find your work wasn’t cited in this note, please feel free to let me know.

Although I am by no means an data mining programming and Big Data expert, I decided that it would be useful for me to share what I learned about PySpark programming in the form of easy tutorials with detailed example. I hope those tutorials will be a valuable tool for your studies.

The tutorials assume that the reader has a preliminary knowledge of programing and Linux. And this document is generated automatically by using sphinx.

BTW, I successfully brought git output format into Sphnix in this repository. You need to install

and more details can be found from the following reference:



Now, the

function for github pages can be easily solved by add an empty file
to your docs folder. I add the following piece of code in my
to add it automatically:
    # add .nojekyll file to fix the github pages issues
    nojekyll_path = os.path.join(outdir, '.nojekyll')
    if not os.path.exists(nojekyll_path):
        nojekyll = open(nojekyll_path,'a')

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.