HIPI: Hadoop Image Processing Interface
HIPI (Hadoop Image Processing Interface) is a library designed to provide efficient and high-throughput image processing in the Apache Hadoop MapReduce parallel programming framework. It also provides support for OpenCV.
For more detail about what HIPI is, see the Main HIPI Webpage.
To stay in touch with the HIPI development community join the HIPI Users Group.
HIPI works with a standard installation of the Apache Hadoop Distributed File System (HDFS) and MapReduce. HIPI has been tested with Hadoop version 2.7.1.
If you haven't already done so, download and install Hadoop by following the instructions on the official Apache Hadoop website. A very useful resource is their Quickstart Guide, in particular, the Single Cluster Setup for first-time users.
Ensure that the Hadoop bin directory is in your system path:
$> which hadoop /usr/local/bin/hadoop
The HIPI distribution uses the Gradle build automation system for code compilation and assembly. HIPI has been tested with Gradle version 2.5:
gradleon your system and verify that it is in your path as well:
$> which gradle /usr/local/bin/gradle
Clone the latest HIPI distribution from GitHub:
$> git clone [email protected]:uvagfx/hipi.git
From the HIPI root directory, simply run
gradleto build the HIPI library along with the complete set of tools and example programs:
$> gradle :checkJavaVersion Found Java 1.8. :core:compileJava :core:processResources :core:classes :core:jar :tools:downloader:compileJava :tools:downloader:processResources :tools:downloader:classes :tools:downloader:jar :tools:dumpHib:compileJava :tools:dumpHib:processResources :tools:dumpHib:classes :tools:dumpHib:jar ... :install
Finished building the HIPI library along with all tools and examples.
Total time: 2.058 secs
If the build fails, first carefully review the steps above. If you are convinced that you are doing everything correctly and that you've found an issue with the HIPI distribution or documentation please post a question to the HIPI Users Group or use the Issue Tracker to file a bug report.
After the build successfully finishes, you may want to inspect the
settings.gradlefile in the root directory along with the
build.gradlefiles in the root, core, and tools directories in order to familiarize yourself with the various build targets. If you're new to Gradle, we recommend reading the Gradle Java Tutorial. For example, to build only the tools/hibImport tool from scratch:
$> gradle clean tools:hibImport:jar :core:clean ... :core:compileJava :core:processResources UP-TO-DATE :core:classes :core:jar :tools:hibImport:compileJava :tools:hibImport:processResources UP-TO-DATE :tools:hibImport:classes :tools:hibImport:jar
Total time: 1.197 secs
HIPI is now installed on your system. To learn about future updates to the HIPI distribution you should join the HIPI Users Group and watch this repository. You can always obtain the latest version of HIPI on the release branch with the following command:
$> git pull origin release From github.com:uvagfx/hipi * branch release -> FETCH_HEAD Already up-to-date.
Also, you can experiment with the
developmentbranch, which often contains new features that have not yet been integrated into the main release branch. Note that the
developmentbranch is generally less stable than the
Be sure to check out the HIPI Tools and Example Programs to learn more about HIPI.