Tika and Computer Vision

This page describes how to make use of Object (Visual) Recognition capability of Apache Tika. TIKA-1993 introduced a new parser to perform object recognition on images. Visit TIKA-1993 issue on Jira or pull request on Github to read the related conversation. Continue reading to get Tika up and running for object recognition.

Currently, Tika utilises Inception-V3 model from Tensorflow for recognising objects in the JPEG images.

Tika and Tensorflow Image Recognition

Tika has two different ways of bindings to Tensorflow:

  1. Using Commandline Invocation -- Recommended for quick testing, not for production use
  2. Using REST API -- Recommended for production use

1. Tensorflow Using Commandline Invocation

Pros of this approach:

Cons:

Step 1. Install the dependencies

To install tensorflow, follow the instructions on the official site here for your environment. Unless you know what you are doing, you are recommended to follow pip installation.

To test the readiness of your environment :

If the above command prints the message "OK", then the requirements are satisfied.

Step 2. Create a Tika-Config XML to enable Tensorflow parser.

Here is an example:

   1 <properties>
   2     <parsers>
   3         <parser class="org.apache.tika.parser.recognition.ObjectRecognitionParser">
   4             <mime>image/jpeg</mime>
   5             <params>
   6                 <param name="topN" type="int">2</param>
   7                 <param name="minConfidence" type="double">0.015</param>
   8                 <param name="class" type="string">org.apache.tika.parser.recognition.tf.TensorflowImageRecParser</param>
   9             </params>
  10         </parser>
  11     </parsers>
  12 </properties>

Description of parameters :

Param Name Type Meaning Range Example
topN int Number of object names to output a non-zero positive integer 1 to receive top 1 object name
minConfidence double Minimum confidence required to output the name of detected objects [0.0 to 1.0] inclusive 0.9 for outputting object names iff at least 90% confident
class string Class that implements object recognition functionality constant string org.apache.tika.parser.recognition.tf.TensorflowImageRecParser

Step 3: Demo

To use the vision capability via Tensorflow, just supply the above configuration to Tika.

For example, to use in Tika App (Assuming you have tika-app JAR and it is ready to run):

      $ java -jar tika-app/target/tika-app-1.14-SNAPSHOT.jar \
         --config=tika-parsers/src/test/resources/org/apache/tika/parser/recognition/tika-config-tflow.xml \
         https://upload.wikimedia.org/wikipedia/commons/f/f6/Working_Dogs%2C_Handlers_Share_Special_Bond_DVIDS124942.jpg

The input image is:

Germal Shepherd with Military

And, the top 2 detections are:

   1 ...
   2 <meta name="OBJECT" content="German shepherd, German shepherd dog, German police dog, alsatian (0.36203)"/>
   3 <meta name="OBJECT" content="military uniform (0.13061)"/>
   4 ...

2. Tensorflow Using REST Server

This is the recommended way for utilizing visual recognition capability of Tika. This approach uses Tensorflow over REST API. To get this working, we are going to start a python flask based REST API server and tell tika to connect to it. All these dependencies and setup complexities are isolated in docker image.

Requirements :

Step 1. Setup REST Server

You can either start the REST server in an isolated docker container or natively on the host that runs tensorflow.

a. Using docker (Recommended)

   1 cd tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/ 
   2 # alternatively, if you do not have tika's source code, you may simply wget the 'InceptionRestDockerfile' from github link 
   3 docker build -f InceptionRestDockerfile -t inception-rest-tika .
   4 docker run -p 8764:8764 -it inception-rest-tika

Once it is done, test the setup by visiting http://localhost:8764/inception/v3/classify?topk=2&url=https://upload.wikimedia.org/wikipedia/commons/f/f6/Working_Dogs%2C_Handlers_Share_Special_Bond_DVIDS124942.jpg in your web browser.

Sample output from API:

{
  "confidence": [
    0.3620266020298004,
    0.13061314821243286
  ],
  "classnames": [
    "German shepherd, German shepherd dog, German police dog, alsatian",
    "military uniform"
  ],
  "classids": [
    211,
    866
  ],
  "time": {
    "read": 165,
    "units": "ms",
    "classification": 470
  }
}

b. Without Using docker

If you chose to setup REST server without a docker container, you are free to manually install all the required tools specified in the docker file.

Note: docker file has setup instructions for Ubuntu, you will have to transform those commands for your environment.

   1    python tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/inceptionapi.py  --port 8764

Step 2. Create a Tika-Config XML to enable Tensorflow parser.

Here is an example:

<properties>
    <parsers>
        <parser class="org.apache.tika.parser.recognition.ObjectRecognitionParser">
            <mime>image/jpeg</mime>
            <params>
                <param name="topN" type="int">2</param>
                <param name="minConfidence" type="double">0.015</param>
                <param name="class" type="string">org.apache.tika.parser.recognition.tf.TensorflowRESTRecogniser</param>
            </params>
        </parser>
    </parsers>
</properties>

Description of parameters :

Param Name Type Meaning Range Example
topN int Number of object names to output a non-zero positive integer 1 to receive top 1 object name
minConfidence double Minimum confidence required to output the name of detected objects [0.0 to 1.0] inclusive 0.9 for outputting object names iff at least 90% confident
class string Name of class that Implements Object recognition Contract constant string org.apache.tika.parser.recognition.tf.TensorflowRESTRecogniser
healthUri URI HTTP URL to check availability of API service any HTTP URL that gets 200 status code when available http://localhost:8764/inception/v3/ping
apiUri URI HTTP URL to POST image datat any HTTP URL that returns data in the JSON format as shown in the sample API output http://localhost:8764/inception/v3/classify?topk=10

Step 3. Demo

This demo is same as the Commandline Invocation approach, but this is faster and efficient

   $ java -jar tika-app/target/tika-app-1.14-SNAPSHOT.jar \
    --config=tika-parsers/src/test/resources/org/apache/tika/parser/recognition/tika-config-tflow-rest.xml \
    https://upload.wikimedia.org/wikipedia/commons/f/f6/Working_Dogs%2C_Handlers_Share_Special_Bond_DVIDS124942.jpg

The input image is:

Germal Shepherd with Military

And, the top 2 detections are:

   1 ...
   2 <meta name="OBJECT" content="German shepherd, German shepherd dog, German police dog, alsatian (0.36203)"/>
   3 <meta name="OBJECT" content="military uniform (0.13061)"/>
   4 ...

Changing the default topN, API port or URL

To change the defaults, update the parameters in config XML file accordingly

Here is an example scenario:

Run REST API on port 3030, and get top 4 object names if the confidence is above 10%. You may also change host to something else than 'localhost' if required.

Example Config File

<properties>
    <parsers>
        <parser class="org.apache.tika.parser.recognition.ObjectRecognitionParser">
            <mime>image/jpeg</mime>
            <params>
                <param name="topN" type="int">4</param>
                <param name="minConfidence" type="double">0.1</param>
                <param name="class" type="string">org.apache.tika.parser.recognition.tf.TensorflowRESTRecogniser</param>
                <param name="healthUri" type="uri">http://localhost:3030/inception/v3/ping</param>
                <param name="apiUri" type="uri">http://localhost:3030/inception/v3/classify?topk=4</param>
            </params>
        </parser>
    </parsers>
</properties>

To Start the service on port 3030:

Using Docker:

Without Using Docker:

python tika-parsers/src/main/resources/org/apache/tika/parser/recognition/tf/inceptionapi.py  --port 3030


Questions / Suggestions / Improvements / Feedback ?

  1. If it was useful, let us know on twitter by mentioning @ApacheTika

  2. If you have questions, let us know by using Mailing Lists

  3. If you find any bugs, use Jira to report them

TikaAndVision (last edited 2016-08-14 22:58:13 by ThammeGowda)