Differences between revisions 5 and 6
Revision 5 as of 2007-09-04 21:01:48
Size: 4588
Editor: 171
Comment: add my email as a source of help.
Revision 6 as of 2007-09-05 20:28:31
Size: 4592
Editor: 128
Deletions are marked like this. Additions are marked like this.
Line 30: Line 30:
4) Unzip the test-files.zip into SOLR_HOME/test/test-files/. These are various test files for running the included unit tests. 4) Unzip the test-files.zip into SOLR_HOME/src/test/test-files/. These are various test files for running the included unit tests.

Updating a Solr Index with Rich Documents such as PDF and MS Office

Solr has an extensible DocumentHandler architecture that allows you to feed it XML and CSV documents. There is now a patch file available as part of [https://issues.apache.org/jira/browse/SOLR-284 SOLR-284] that adds support for parsing rich binary formats.

This page talks about how to get started using this patch. If you like it, please [https://issues.apache.org/jira/secure/ViewVoters!default.jspa?id=12372848 vote] for it on the JIRA issue tracker so we can get it added to the Solr codebase!

Any additions, enhancements, or questions, please email me at epugh at opensourceconnections.com, or just update the JIRA issue!



Solr 1.3 is the current development version, and what the code was developed against. The Rich Document request handler needs to be configured in solrconfig.xml This should already be present in the example solrconfig.xml if you follow the Install directions

  <!-- Rich document update handler, loaded on demand -->
  <requestHandler name="/update/rich" class="solr.RichDocumentRequestHandler" startup="lazy">

How to Install

1) You need a couple patch files and zips of source and test code that are attached to the JIRA issue at https://issues.apache.org/jira/browse/SOLR-284.

2) Download the libs.zip, rich.patch, test-files.zip, source.zip, and test.zip files.

3) Unzip the libs.zip into SOLR_HOME/lib. These are the jar's required for parsing the rich documents, using PDFBox and POI.

4) Unzip the test-files.zip into SOLR_HOME/src/test/test-files/. These are various test files for running the included unit tests.

5) Apply the rich.patch to your source. Rich.patch has tweaks that add the solr.RichDocumentRequestHandler to your solrconfig.xml files.

6) Copy the contents of source.zip into SOLR_HOME/src/java/org/apache/solr/handler

7) Copy the contents of test.zip into SOLR_HOME/src/test/org/apache/solr/handler

8) Run ant test to verify everything is working!

Methods of uploading Binary records

Binary records may be uploaded to Solr by sending the data to the /solr/update/rich URL. All of the normal methods for [SolrContentStreams uploading content] are supported.


These examples assume you have run ant example to generate the example app first and have started Solr up using java -jar start.jar.

There is a sample Word file at src/test/test-files/complex.doc that may be used to add a Word document to the Solr example server.

Example of using HTTP-POST to send the PDF data over the network to the Solr server in browser:

curl http://localhost:8983/solr/update/rich?stream.type=pdf --data-binary @simple.pdf -H 'Content-type:text/plain; charset=utf-8'

Uploading a binary file can be more efficient than sending it over the network via HTTP. Remote streaming must be enabled for this method to work. See the following line in solrconfig.xml, change it to enableRemoteStreaming="true", and restart Solr.

  <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />

The following request will cause Solr to directly read the input file:

URL in browser to add document
#NOTE: The full path, or a path relative to the CWD of the running solr server must be used.

URL in browser to get results



Specifies what format the document is. It may be pdf, doc, ppt, or xls.

Example: stream.type=ppt


The name of the field defined in schema.xml to store the contents of the file in.

Example: stream.fieldname=text


A list of metadata fields to also index with this document.

example: fieldnames=author,subject&author=Bill&subject=fiction


Commit changes after all records in this request have been indexed. The default is commit=false to avoid the potential performance impact of frequent commits.


If true (the default), overwrite documents based on the uniqueKey field declared in the solr schema.


There is no way to provide document or field index-time boosts, however many indicies do not utilize that feature.

UpdateRichDocuments (last edited 2009-09-20 22:04:42 by localhost)