Added Norconex Collectors to "Crawlers And Connectors" section. Sorted the entries.
Removed recently added "Apache" prefix to Nutch in Crawler and Connectors since other Apache products listed did not have it (consistency).
|Deletions are marked like this.||Additions are marked like this.|
|Line 17:||Line 17:|
|* [[http://lucene.apache.org/nutch/|Apache Nutch]] (web) [[http://wiki.apache.org/nutch/NutchTutorial|Solr Info (included as part of the Nutch Tutorial)]]|
|Line 26:||Line 25:|
|* [[http://lucene.apache.org/nutch/|Nutch]] (web) [[http://wiki.apache.org/nutch/NutchTutorial|Solr Info (included as part of the Nutch Tutorial)]]|
This page documents that relate to the broader Solr Ecosystem of Search and Information Retrieval and may be of interest to Solr users. Some of these projects may have features that can be used directly in conjunction with Solr. You may also be interested in the IntegratingSolr page which documents client libraries and projects for integrating Solr with other applications not specific to Search/IR.
Solr distributions / forks
Lucidworks Fusion - commercially supported platform extending and enhancing Solr with various enterprise demanded capabilities, including security, connectors, indexing and querying pipelines, signal handling, analytics, etc.
Solandra - A tight integration of Solr and Cassandra. The result is Solr with the awesome scalability properties of Cassandra.
There are numerous ways to bring data into Solr. Many people roll their own solution or use the DataImportHandler
Crawlers And Connectors
Web, email, and file crawlers (alphabetically).
Aperture (web, email, file)
DataImportHandler (email, file)
- Presently, more of a framework for a crawler.
Pipelines / Document Processing
Frameworks for flexible document processing. See DocumentProcessing for more background and criteria for a proposal. Some crawlers/connectors have their own pipeline capability and they are not repeated here.
Aspire (by Search Technologies) - integrates with Solr. Not open-source but free.
Hydra (by Findwise) - integrates with Solr.
- ETL (Extract Transform Load) -- many are applicable; these are a couple notable ones:
A common problem amongst the ETLs is that each step in the pipeline accepts and emits records in a fixed flat schema, they don't support dynamic name-value pairs. And these are not document oriented; if you want to pass a DOM of some kind then you serialize it into a field. However, the ETLs are all far more mature than nascent document or XML oriented pipelines.
- ESBs (Enterprise Service Buses) -- not listed; various
Generating the Lucene/Solr Index
Tools or services for monitoring Solr-specific performance metrics.