Apache Nutchâ„Ē

Nutch is a highly extensible, highly scalable, matured, production-ready Web crawler which enables fine grained configuration and accomodates a wide variety of data acquisition tasks.

Baseline

Scalable

Relying on Apache Hadoopâ„Ē data structures, Nutch is great for batch processing large data volumes but can also be tailored to smaller jobs.

Typography

Pluggable

Out of the box Nutch offer powerful plugins i.e., parsing with Apache Tikaâ„Ē, indexing with Apache Solrâ„Ē, Elasticsearch and more!

Minimalism

Extensible

Provides intuitive and stable interfaces for popular functions i.e., Parsers, HTML Filtering, Indexing and Scoring for custom implementations.