Need help with collector-http?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

Norconex
131 Stars 66 Forks Apache License 2.0 941 Commits 82 Opened issues

Description

Norconex HTTP Collector is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.

Services available

!
?

Need anything else?

Contributors list

Norconex HTTP Collector

Norconex HTTP Collector

Norconex HTTP Collector is a full-featured web crawler (or spider) that can manipulate and store collected data into a repositoriy of your choice (e.g. a search engine). It very flexible, powerful, easy to extend, and portable. Can be used command-line with file-based configuration on any OS, or can be embedded into Java applications using well documented APIs.

Visit the web site for binary downloads and documentation:

https://opensource.norconex.com/collectors/http/

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.