Tika and Computer Vision powered by Deeplearning4j(DL4J)

This page describes a way to make Tika perform image recognition. Tika has many implementations of image recognition parsers. Specifically, this page provides information for an implementation powered by Deeplearning4j, InceptionNet-V3 model pre-trained on ImageNet dataset and now with TIKA-2298, the VGG-16 model. Both modesl can detect a thousand different objects in the images.

The advantage of this particular setting is, this implementation runs inside a Java Virtual Machine (JVM) stack without dependence on any external services. So it is perfect for the users who are trying to run image recognition on a distributed setup like Apache Hadoop or Apache Spark.

Note:

  1. This is a work in progress. Inception-V3 was added in Tika 1.15 and VGG-16 in 1.16.
  2. At the time of writing, Tika 1.16 was not released. You have to [clone Tika repository](https://github.com/apache/tika) and do mvn clean install.

  3. The rest of the page uses version 1.15-SNAPSHOT, however, if you are reading this after release, please use 1.15, 1.16 or newer version.

Java/Groovy/Scala example

For maven users: Add Tika-parsers and tika-dl to your project

Here is an example for Apache Maven users:

   1     <dependencies>
   2         <dependency>
   3             <groupId>org.apache.tika</groupId>
   4             <artifactId>tika-parsers</artifactId>
   5             <version>1.15-SNAPSHOT</version>
   6         </dependency>
   7         <dependency>
   8             <groupId>org.apache.tika</groupId>
   9             <artifactId>tika-dl</artifactId>
  10             <version>1.15-SNAPSHOT</version>
  11         </dependency>
  12     </dependencies>

A configuration, tika-config.xml, to activate Inception-V3 image recognition model:

   1 <properties>
   2   <parsers>
   3     <parser class="org.apache.tika.parser.recognition.ObjectRecognitionParser">
   4       <mime>image/jpeg</mime>
   5       <params>
   6         <param name="topN" type="int">10</param>
   7         <param name="minConfidence" type="double">0.015</param>
   8         <param name="class" type="string">org.apache.tika.dl.imagerec.DL4JInceptionV3Net</param>
   9       </params>
  10     </parser>
  11   </parsers>
  12 </properties>

A configuration, tika-config.xml, to activate VGG-16 image recognition model:

   1 <?xml version="1.0" encoding="UTF-8"?>
   2 <properties>
   3     <parsers>
   4         <parser class="org.apache.tika.parser.recognition.ObjectRecognitionParser">
   5             <mime>image/jpeg</mime>
   6             <params>
   7                 <param name="topN" type="int">2</param>
   8                 <param name="minConfidence" type="double">0.015</param>
   9                 <param name="class" type="string">org.apache.tika.dl.imagerec.DL4JVGG16Net</param>
  10                 <param name="modelType" type="string">VGG16</param>
  11                 <param name="serialize" type="string">yes</param>
  12             </params>
  13         </parser>
  14     </parsers>
  15 </properties> 

Note: Refer to a later section for customizing the config.

Sample Java code:

   1     //create parser as per desired parser
   2     TikaConfig config;
   3     try (InputStream stream = ImageRecLocal.class.getClassLoader()
   4             .getResourceAsStream("tika-config.xml")){
   5         config = new TikaConfig(stream);
   6     }
   7     Tika parser = new Tika(config);
   8     //sample file
   9     File imageFile = new File("data/gun.jpg");
  10     Metadata meta = new Metadata();
  11     parser.parse(imageFile, meta);
  12     //retrieve objects from the metadata
  13     System.out.println(Arrays.toString(meta.getValues("OBJECT")));
  14     // This should print: [assault_rifle (0.78214), rifle (0.18048), revolver (0.02780)]
  15 

For Python and Tika Server users

  1. Create file with name tika-config.xml by using the content shown above.

  2. Clone tika repository and build the tika project
  3. Add tika-dl to classpath of tika server

   1 alias tika="java -cp tika-server/target/tika-server-1.15-SNAPSHOT.jar:tika-dl/target/tika-dl-1.15-SNAPSHOT-jar-with-dependencies.jar org.apache.tika.server.TikaServerCli "
   2 tika --config=tika-config.xml # Starts server port 9998
   3 

Refer to Tika-python for an example usage.

Large Scale Image Recognition In Spark

Coming soon! It is being tested here https://github.com/thammegowda/tika-dl4j-spark-imgrec

Configuration options

Previously, we have used the three parameters to this parser:

   1 <params>
   2     <param name="topN" type="int">10</param>
   3     <param name="minConfidence" type="double">0.015</param>
   4     <param name="class" type="string">org.apache.tika.dl.imagerec.DL4JInceptionV3Net</param>
   5 </params>

The other important parameters are:

   1     <param name="modelWeightsPath" type="string">VALUE</param>
   2     <param name="modelJsonPath" type="string">VALUE</param>

The VALUE string can be:

  1. File path relative to class loader (note that these could also be inside jar files)
  2. Absolute file path anywhere on the local filesystem
  3. Any remote URL, including HTTP or HTTPS

For example:

  1. <param name="modelWeightsPath" type="string">inception-model-weights.h5</param>

  2. <param name="modelWeightsPath" type="string">/usr/share/apache-tika/models/tikainception-model-weights.h5</param>

  3. <param name="modelWeightsPath" type="string">https://myserver.com/files/apache-tika/models/tikainception-model-weights.h5</param>


Questions / Suggestions / Improvements / Feedback ?

  1. If it was useful, let us know on twitter by mentioning @ApacheTika

  2. If you have questions, let us know by using Mailing Lists

  3. If you find any bugs, use Jira to report them

TikaAndVisionDL4J (last edited 2017-07-06 03:20:06 by ChrisMattmann)