ContentStreams are an abstraction mechanism for accessing streaming data in SolrRequestHandlers.

Overview

When SolrRequestHandlers are accessed using path based URLs the SolrQueryRequest object containing the parameters of the request may also contain a list of ContentStreams containing bulk data for the request.

(The name SolrQueryRequest is a bit misleading: it is involved in all requests, regardless of whether it is a query request or an update request.)

Stream Sources

Currently RequestHandlers can get content streams a variety of ways:

If the contentType is "application/x-www-form-urlencoded" the full POST body is parsed as parameters and inlcuded in the SolrParams.

By default, curl sends a contentType="application/x-www-form-urlencoded" header. If you need to test a SolrRequestHandler content stream, you will need to set the content type with the "-H" flag. For example:

 curl $URL -H 'Content-type:text/xml; charset=utf-8' --data-binary @$f

RemoteStreaming

Remote streaming allows you to send the contents of a URL as a stream to a given SolrRequestHandler. This could be used to send a remote or local file to an update plugin. If remote streaming is enabled, be aware that this allows anyone to send a request to any URL or local file and essentially allow anyone that can send a request directly to solr to view any file that solr can read.

  <!--Make sure your system has authentication before enabling remote streaming!-->
  <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048" />

Example:

curl 'http://localhost:8983/solr/update/csv?stream.file=exampledocs/books.csv&stream.contentType=text/plain;charset=utf-8'

Debugging

The example solrconfig.xml includes a 'dump' handler

  <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" />

This handler simply outputs the contents of the SolrQueryRequest using the specified writer type 'wt'. This is a useful tool to help understand what streams are available to to the RequestHandlers.

ContentStream (last edited 2011-03-22 18:29:12 by YonikSeeley)