Introduction

This tutorial demonstrates how to create a very simple connection between the OODT File Manager and SIS. The connection is made by outputting an RSS feed of data from the File Manager via the REST API and configuring SIS to read in this RSS data.

Please note, this demonstration is very much a work-in-progress! It represents the first steps in making a connection between OODT and SIS. Work is ongoing and captured by issue OODT-402. As work continues it is hoped that a more formal, elegant and robust connection between OODT and SIS will be defined. This page will be updated accordingly to reflect the latest developments.

In the meantime, if you spot any typos, mistakes or feel that there are much better ways of doing things, please feel free to comment or update the wiki, send messages to the mailing lists or post to the OODT-402 ticket! Your comments will always be much appreciated!

Prerequisites / Assumptions

This tutorial assumes that you already have the OODT File Manager, OODT File Manager REST API and SIS webapp installed and running on your machine.

For the purposes of this demonstration, it is best to start with an empty File Manager ‘test’ repository, i.e. no files yet ingested into your repository, as we will be modifying the meta tags for the GenericFile product type.

(In the future, the aim is to define a separate ‘LocationAware’ product type that will not interfere with the GenericFile definition and can therefore be used with existing repositories.)

File Manager Configuration

The first step is to configure ‘Latitude’ and ‘Longitude’ meta tags for the GenericFile type in File Manager. This is achieved by editing the File Manager policy files (by default these are located in ‘/usr/local/oodt/filemgr/policy/core’ after installation).

Firstly, edit the ‘elements.xml’ file to add the following element definitions between the <cas:elements> … </cas:elements> tags:

<element id="urn:oodt:Latitude" name="Latitude">
  <description>Geographical latitude location.</description>
  <dcElement />
</element>

<element id="urn:oodt:Longitude" name="Longitude">
  <description>Geographical longitude location.</description>
  <dcElement />
</element>

Next, edit the ‘product-type-element-map.xml’ file and map the elements to the GenericFile product type by adding the following tags between the <type> … </type> tags:

<element id="urn:oodt:Latitude"/>
<element id="urn:oodt:Longitude"/>

After updating and saving the files, restart File Manager, for example by entering the following command from the File Manager bin directory:

./filemgr restart

Ingesting Sample Files

Create a few sample text files with associated metadata files. Ensure the metadata files contain metadata for the new ‘Latitude’ and ‘Longitude’ fields.

For example, in the /tmp directory create a ‘geodata.txt’ file containing a few arbitrary words of your own choice. Then create a ‘geodata.txt.met’ metadata file containing the following text:

<cas:metadata xmlns:cas="http://oodt.apache.org/ns/cas">

  <keyval>
    <key>Latitude</key>
    <val>10</val>
  </keyval>

  <keyval>
    <key>Longitude</key>
    <val>30</val>
  </keyval>

</cas:metadata>

Ingest the files into your File Manager repository. Below is an example command for ingesting files using the ‘filemgr-client’ tool from the File Manager bin directory for a File Manager instance running on port 9000.

./filemgr-client --url http://localhost:9000 --operation --ingestProduct \
--productName geodata.txt --productStructure Flat \
--productTypeName GenericFile \
--metadataFile file:///tmp/geodata.txt.met \
--refs file:///tmp/geodata.txt

File Manager REST API Webapp Configuration

Add the following tag definitions to the ‘rssconf.xml’ configuration file for the File Manager REST API webapp. By default, ‘rssconf.xml’ is located in the WEB-INF/classes directory of the webapp. The new tag definitions should be inserted between the <cas:rssconf> … </cas:rssconf> tags.

<tag name="geo:lat" source="[Latitude]">
  <attribute name="xmlns:geo" value="http://www.w3.org/2003/01/geo/wgs84_pos#" />
</tag>

<tag name="geo:long" source="[Longitude]">
  <attribute name="xmlns:geo" value="http://www.w3.org/2003/01/geo/wgs84_pos#" />
</tag>

The ‘xmlns:geo’ namespace definitions are required for ‘geo:lat’ and ‘geo:long’ tags to be interpreted correctly. The above inline solution was chosen as a first step to get the connection working, but there is probably a much more elegant way of doing this.

After updating the ‘rssconf.xml’ file, you can test your updates by doing the following:

1) Restart Tomcat, for example by entering the following commands from the Tomcat bin directory:

./shutdown.sh
./startup.sh

2) Navigate to ‘http://<host>/<webapp>/viewRecent?channel=ALL’. Where <host> is the host location for Tomcat and <webapp> is the directory of your File Manager REST API webapp. For example, ‘’.

After navigating to the above location, you should see an RSS output. Depending on your browser, you may have to ‘view source’ to see the XML. You should be able to see your new tags in the output, for example:

<rss xmlns:cas="http://oodt.apache.org/ns/cas" version="2.0">
  <channel>
    <title>ALL</title>
    <link>
      http://localhost:8080/fmprod/rdf/dataset?type=ALL&typeID=null
    </link>
    <description>ALL</description>
    <language>en-us</language>
    <copyright>Copyright 2010: Apache Software Foundation</copyright>
    <pubDate>Mon, 02 Apr 2012 23:24:07 BST</pubDate>
    <category>ALL</category>
    <generator>CAS File Manager</generator>
    <lastBuildDate>Mon, 02 Apr 2012 23:24:07 BST</lastBuildDate>
    <item>
      <title>geodata.txt</title>
      <description>GenericFile</description>
      <link>
        http://localhost:8080/fmprod/data?productID=532f56ef-7d12-11e1-90fa-8523ce23740f
      </link>
      <pubDate>Tue, 03 Apr 2012 00:22:25 BST</pubDate>
      <geo:lat xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">10</geo:lat>
      <geo:long xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">30</geo:long>
      <cas:source>GenericFile</cas:source>
      <source>GenericFile</source>
    </item>
  </channel>
</rss>

SIS Webapp Configuration

Add the URL used in the File Manager REST API to the ‘sis_location_config.xml’ file for the SIS webapp. By default, this is located in the WEB-INF/classes directory of the SIS webapp. Remove all of the other RSS URLs from the file. After editing the file, it should look similar to the following:

<sis:locationConfig xmlns:sis="http://sis.apache.org/ns/sis">
  <capacity>4</capacity>
  <depth>10</depth>
  <url>http://localhost:8080/fmprod/viewRecent?channel=ALL</url>
</sis:locationConfig>

Restart Tomcat to ensure that the changes to the configuration file are detected. [Note: If you have previously run the ‘demo.jsp’ SIS webapp demonstration page, it may be necessary to clear out the stored data from the webapp. This can be done by deleting the folders ‘geodata’ and ‘qtree’ from the main SIS webapp directory inbetween stopping and starting Tomcat. The folders will be recreated by the webapp when the next query is run.]

Running The Demo

To demonstrate the connection we'll use the 'demo.jsp' demo from the SIS web application. Open a web browser and navigate to http://<host>/<webapp>/demo.jsp, where <host> is the Tomcat location and <webapp> is the SIS webapp folder, for example: ‘http://localhost:8080/sis/demo.jsp’.

Enter data in the query form to return an area that should encompass the points defined in your metadata files. For example, the following bounding box query should return an area large enough to include the point defined in the ‘geodata.txt.met’ file:

Lower Left: Latitude: 0, Longitude: 0
Upper Right: Latitude: 50, Longitude: 50

Press the ‘Query’ button and view the map onscreen. All being well, your File Manager files should be visible on the map, as in the following screenshot. Click on the markers on the map to display information about them and confirm that it is indeed your files that are being displayed: