Differences between revisions 1 and 2
Revision 1 as of 2013-03-20 18:31:40
Size: 393
Comment:
Revision 2 as of 2015-06-13 18:24:26
Size: 893
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
The bin/crawl script gives more command during a crawl. Instead of using org.apache.nutch.crawl.Crawl class, it uses individual steps (inject->generate->fetch->parse->updatedb) during a crawl. It is recommended to use this instead of using the [[bin/nutch crawl]] command. = Description =
The bin/crawl script gives more command during a crawl. It uses individual steps (inject->generate->fetch->parse->updatedb) during a crawl.
Line 3: Line 4:
= Usage =
== Nutch 1.X ==
{{{
     Usage: crawl [-i|--index] [-D "key=value"] <Seed Dir> <Crawl Dir> <Num Rounds>
        -i|--index Indexes crawl results into a configured indexer
        -D A Java property to pass to Nutch calls
        Seed Dir Directory in which to look for a seeds file
        Crawl Dir Directory where the crawl/link/segments dirs are saved
        Num Rounds The number of rounds to run this crawl for
     Example: bin/crawl -i -D solr.server.url=http://localhost:8983/solr/ urls/ TestCrawl/ 2
}}}

== Nutch 2.x ==

= Need Assistance ? =

Description

The bin/crawl script gives more command during a crawl. It uses individual steps (inject->generate->fetch->parse->updatedb) during a crawl.

Usage

Nutch 1.X

     Usage: crawl [-i|--index] [-D "key=value"] <Seed Dir> <Crawl Dir> <Num Rounds>
        -i|--index      Indexes crawl results into a configured indexer
        -D              A Java property to pass to Nutch calls
        Seed Dir        Directory in which to look for a seeds file
        Crawl Dir       Directory where the crawl/link/segments dirs are saved
        Num Rounds      The number of rounds to run this crawl for
     Example: bin/crawl -i -D solr.server.url=http://localhost:8983/solr/ urls/ TestCrawl/  2

Nutch 2.x

Need Assistance ?

Please message us in the user-mailing list if you find any issues

bin/crawl (last edited 2015-06-13 18:24:26 by LewisJohnMcgibbney)