ContentStream

ContentStreams are an abstraction mechanism for accessing streaming data in SolrRequestHandlers.

1. Overview
2. Stream Sources
3. RemoteStreaming
4. Debugging

Overview

When SolrRequestHandlers are accessed using path based URLs the SolrQueryRequest object may that contains the parameters or the request may also contain a list of ContentStreams containing bulk data for the request.

(The name SolrQueryRequest is a bit misleading: it is involved in all requests, regardless of whether it is a query request or an update request.)

Stream Sources

Currently RequestHandlers can get content streams a variety of ways:

• For Multipart file upload, each file is passed as a stream.

• For POST requests where the content-type is not "application/x-www-form-urlencoded", the raw POST body is passed as a stream.

• The contents of parameter "stream.body" is passed as a stream.

• If remote streaming is enabled, the contents of each "stream.url" and "stream.file" parameters are fetched and passed as a stream

If the contentType is "application/x-www-form-urlencoded" the full POST body is parsed as parameters and inlcuded in the SolrParams.

By default, curl sends contentType="application/x-www-form-urlencoded". If you need to test a SolrRequestHandler content stream, you wil need to set the content type with the "-H" flag. For example:

 curl $URL -H 'Content-type:text/xml; charset=utf-8' --data-binary @$f  

RemoteStreaming

Remote streaming allows you to send the contents of a URL as a stream to a given SolrRequestHandler. This could be used to send a remote or local file to an update plugin. For security reasons, remote streaming is disabled in the example solrconfig.xml. If you enable streaming, be aware that this allows anyone to send a request to any URL or local file. If dump is enabled, it will essentially let anyone to view any file on your system.

  <!--Make sure your system has authentication before enabling remote streaming!-->
  <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048" />

Debugging

The example solrconfig.xml includes a 'dump' handler

  <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" />

This handler simply outputs the contents of the SolrQueryRequest using the specified writer type 'wt'. This is a useful tool to help understand what streams are available to to the RequestHandlers.