Differences between revisions 7 and 8
Revision 7 as of 2008-07-29 10:15:09
Size: 9852
Editor: 86
Comment:
Revision 8 as of 2008-10-03 14:49:31
Size: 9794
Editor: d86-33-64-61
Comment:
Deletions are marked like this. Additions are marked like this.
Line 25: Line 25:
The best way to get an idea how a widget may look like and how it can be integrated into html is to explore the documented source code and the test*.html example pages at [http://solrstuff.org/svn/solrjs/trunk]. Use the testsolr that includes all necessary patches, start it with "java -jar start" and point your browser to the example html files inside the "test" directory. The best way to get an idea how a widget may look like and how it can be integrated into html is to explore the documented source code and the test*.html example pages at [http://solrstuff.org/svn/solrjs/trunk]. Use "ant testsolr-import" to get some testdata and "ant testsolr start" to start the included testserver.

About

SolrJs is a javascript client library that is topic of a 2008 google summer of code project. It is currently work in progress, this page is intended to show the project's status, some technical documentation and a collection of thoughts about future features. The current trunk is accessible at [http://solrstuff.org/svn/solrjs/trunk]

Architectural Overview

The library is written using the javascript toolkit jQuery [http://jquery.com/] (Version 1.2.5). After distribution, SolrJS is included in one additional javascript file "solrjs.js". The idea is to create several (reusable and extensible) "widgets" that represent solr queries. A widget is a javascript object that is responsible for creating the according solr query as well as render the result from the server to html. One manager object acts as a container that holds these widgets, performs the actual query using jQueries getJSON [http://docs.jquery.com/Ajax/jQuery.getJSON#urldatacallback] method. This method creates a dynamic script tag, making cross-domain solr requests possible.

There are two base types of widgets:

  • client side widgets: They get a Javascript JSON response from the server and render the needed html for the widget using jQuery javascript code.
  • server side widgets: They move the rendering logic to the server, using the VelocityResponseWriter (see [https://issues.apache.org/jira/browse/SOLR-620]. The widget then only "copies" the html reponse from the server into the target div.

To make the framework more clearly arranged, the every js "class" lies inside a single JS file, they are organized in "packages" like in a Java Application:

  • core: The manager and other base classes.
  • server: server side widgets
  • client: client side widgets

For distribution, there is a small ant build script provided taht creates one single js file out of these snippets. Just run "ant dist". In the future, there might be some js compression steps included.

Examples and source code

The best way to get an idea how a widget may look like and how it can be integrated into html is to explore the documented source code and the test*.html example pages at [http://solrstuff.org/svn/solrjs/trunk]. Use "ant testsolr-import" to get some testdata and "ant testsolr start" to start the included testserver.

Despite this, I'll try to describe some important implementation further on this page.

Implementation

Javascript matters

All SolrJS objects are created inside the jQuery.solrjs namspace. Given JQueries support for overriding the $() function, it is easily possible to use SolrJS alongside other javascript toolkits like prototype or custom javascript code.

Just include the following includes in the header section of your html:

<script src="jquery-1.2.5.js"/>
<script src="solrjs.js"/>
<script> var $sj = jQuery.noConflict(); </script> 

After that, all SolrJS objects are accessible using $sj.solrjs.*

The manager object

The manager object acts as "Controller" for the user input. It holds a state of the current solr query. The current query is represented by QueryItems objetcs. A query item simply is a field:value pair. A user may select or deselect multiple items. In the current work in progress, all query itemsa are concatinated using "AND" operators. More complex query generation may be implemented in the future. The manager currently provides the following methods:

  • addWidget: add a new widget
  • addQueryItems: add a list of field:value pairs that restrict the current query
  • removeQueryItems: remove the given items from the current query
  • doRequest(start): performs the actual solr requests
  • doRequestAll(): clears queryItems and requests all documents

How a query request works

Every widget implements a "getSolrUrl" method and performs its own http request. It also has to provide a "handleResult" method that gets the json data response to render. So if a user changes the selection:

  • manager.doRequest(startindex) is called
  • a q=... parameter is created by the manager according to the current selection
  • the call is delegated to all the widgets. They may extend the query url in the getSolrUrl method (eg. adding facet=true..) and then make the http call to solr.
  • json or html data is returned to the widget and widget.handleResponse(data) is called.

Widget Inheritance

A main goal of the implementation was to create clean easy to understand code that can be easily extended by other developers. So I introduced a small OO inheritance pattern inspired by this thread [http://groups.google.com/group/jquery-dev/msg/12d01b62c2f30671] and created the factory method:

jQuery.solrjs.createClass = function(subClassName, baseClassName, subClass)

An "AbstractWidget" class is provided that acts as base class for all further widget implementations. In the source code, all methods that should be implemented by the child class are documented and marked as "abstract methods". An example syntax for a simple widget may be:

/*
 * Simple faceted view for one field.
 * 
 * @param "id": the widgetId
 * @param "target": the target html element in jquery notation
 * @param "fieldName": The facet field name
 */
jQuery.solrjs.createClass ("SimpleFacetWidget", "AbstractWidget", { 
  
  getSolrUrl : function(start) { 
    return "&facet=true&facet.field=" + this.fieldName;
  },

  handleResult : function(data) { 
    var values = data.facet_counts.facet_fields[this.fieldName];         
    jQuery(this.target).html("");
                
    for (var i = 0; i < values.length; i = i + 2) {
      var items =  "[new jQuery.solrjs.QueryItem({field:'" + this.fieldName + "',value:'" +  values[i] + "'})]";
      var label = values[i] + "(" + values[i+1] + ")";          
      jQuery('<a/>').html(label).attr("href","javascript:solrjsManager.selectItems('" + this.id + "'," + items + ")").appendTo(this.target);
        jQuery('<br/>').appendTo(this.target);
      }
  },

  handleSelect : function(data) { 
    jQuery(this.target).html(this.selectedItems[0].value);
    jQuery('<a/>').html("(x)").attr("href","javascript:solrjsManager.deselectItems('" + this.id + "')").appendTo(this.target);
  },

  handleDeselect : function(data) { 
    // do nothing
  }        
});

To add this widget to your html page, just create this script tag:

<script>
  var solrjsManager; 
  $sj(document).ready(function(){ 
    solrjsManager = new $sj.solrjs.Manager({solrUrl:"http://localhost:8983/solr/select"});   
    solrjsManager.addWidget(new $sj.solrjs.SimpleResultWidget({id:"result", target:"#result", rows:20}));       
    solrjsManager.addWidget(new $sj.solrjs.SimpleFacetWidget({id:"cat", target:"#cat", fieldName:"cat"}));
    solrjsManager.doRequestAll(); 
  });
</script>

Example for an easily customized widget: ExtensibleResultWidget

The ExtensibleResultWidget is a showcase of a flexible widget. It provides 2 abstract methods:

  • renderDataItem
  • renderPageStatus

After the document result for the current page is returned, these methods are called and the user is able to use custom javascript code to render the result properly.

javascript:

var resultWidget = new $sj.solrjs.ExtensibleResultWidget({
  id:"result", 
  target:"#result", 
  rows:20,
  renderDataItem : function(item) { 
    jQuery('<div/>').html(item.id).appendTo(this.target);
    // more custom code for rendering one single result item
  }, 
  renderPageStatus : function(first, last, total) { 
    // custom code for rendering the pagination status                                  
    jQuery('#pageStatus_start').html(first + "");
    jQuery('#pageStatus_last').html(last + "");
    jQuery('#pageStatus_total').html(total + "");
    var navigation = $sj('#navigation').html("");
    var pageSize = last - first + 1;
    var lastpageFirst = Math.ceil(total / pageSize) * pageSize - pageSize;
    // create navigation buttons for paging
  }
});
solrjsManager.addWidget(resultWidget);          


html:

<body>
  <div id="pageStatus">
        Showing <span id="pageStatus_start"></span> to <span id="pageStatus_last"></span> of <span id="pageStatus_total"></span>
  </div>
  <div id="navigation"></div>
  <div id="result"></div>
</body>
        
                        

Ideas for widgets

Some of this widgets will be implemented during this years Summer of Code and weill be provided alongode the framework.

  • Server and client side result views.
  • Facet widget for datefields using a timeline chart
  • Facet widget for datefields using a javascript calendar
  • Facet widget for hierarchical fields represented as a tree (category_0, category_1,..)
  • fulltext search
  • autocomplete box for facet values
  • Geographical selection using google maps or the like.
  • Facet widgets that depend on each other
  • A single document view that shows similar documents
  • ......

Feel free to extend this list.

Security concerns

As we have to access the solr web app directly to get the json data, a discussion about security was raised on the user list. See Ryan's posting about a single, restricted request handler that only allows selects (NO updates and deletes) [http://www.nabble.com/Announcement-of-Solr-Javascript-Client-to17462581.html#a17462581]. This "ProxyServlet" even could check and modify some parameters to avoid dos attacks or similar (eg. restricting &rows= to a reasonable maximum). Using the jQuery-getJSON method makes it possible to create cross site json requests, so we even can attach this restricted read-only url path to the solr webapp. Every user may then point the javascript client to eg.http://solrserver/readonly/

SolrJS (last edited 2013-05-06 17:39:12 by 59)