shifu

by ShifuML

ShifuML / shifu

An end-to-end machine learning and data mining framework on Hadoop

200 Stars 101 Forks Last release: over 1 year ago (0.12.0) Other 2.5K Commits 24 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Shifu

Build Status
Maven Central

Download

Please download latest shifu here.

Getting Started

After shifu downloading, build your first model with Shifu tutorial. More details about shifu can be found in our wiki pages.

What is Shifu?

Shifu is an open-source, end-to-end machine learning and data mining framework built on top of Hadoop. Shifu is designed for data scientists, simplifying the life-cycle of building machine learning models. While originally built for fraud modeling, Shifu is generalized for many other modeling domains.

One of Shifu's pros is an end-to-end modeling pipeline in machine learning. With only configurations settings, a whole machine pipeline can be built and model can be much more easy to develop and push to production. The pipeline defined in Shifu is in below:

Shifu Pipeline

Shifu provides a simple command-line interface for each step of the model building process, including

Shifu’s fast Hadoop-based, distributed neural network / logistic regression / gradient boosted trees training can reduce model training time from days to hours on TB data sets. Shifu integrates with Pig workflows on Hadoop, and Shifu-trained models can be integrated into production code with a simple Java API. Shifu leverages Pig, Akka, Encog and other open source projects.

Guagua, an in-memory iterative computing framework on Hadoop YARN is developed as sub-project of Shifu to accelerate training progress.

More details about shifu can be found in our wiki pages

Conference

Contributors

Google Group

Please join Shifu group if questions, bugs or anything else.

Copyright and License

Copyright 2012-2019, PayPal Software Foundation under the Apache License.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.