by Norconex

Norconex / collector-http

Norconex HTTP Collector is a flexible web crawler for collecting, parsing, and manipulating data fro...

128 Stars 65 Forks Last release: 10 months ago (norconex-collector-http-2.9.0) Apache License 2.0 910 Commits 28 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Norconex HTTP Collector

Norconex HTTP Collector

Norconex HTTP Collector is a full-featured web crawler (or spider) that can manipulate and store collected data into a repositoriy of your choice (e.g. a search engine). It very flexible, powerful, easy to extend, and portable. Can be used command-line with file-based configuration on any OS, or can be embedded into Java applications using well documented APIs.

Visit the web site for binary downloads and documentation:

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.