Getting Torque To Work

Although Torque is a great tool and remarkably effective once you get it going, many people have trouble setting it up. The documentation seems particularly weak in this area (possibly because there are so many different databases, drivers, and parameters to consider.)

This document is an attempt to help people figure out what's going wrong with their setup by listing the problems and solutions that other people have had.

PLEASE, PLEASE, PLEASE!! If you have had problems setting up Torque and solved them, add your experiences to the list. The more people who contribute, the better the document!


Getting Torque to generate the schema and object files

This is a listing of the typical problems people have trying to get Torque to generate their object files and schema files.

  1. My problem was that I couldn't generate the java source code by running ant -f build-torque.xml. Everything else was working perfect, I got no error messages from Ant, sql code was generated and I could run that to generate my mysql database and tables and everything else. My mistake was that I wasn't setting the right name for my database schema xml file. If the project name in build.properties is "pitchers" ("torque.project = pitchers"), the schema xml file (in the schema folder) needs to be named as "pitchers-schema.xml" i.e in the format "project-schema.xml"

Might be a dumb mistake but that's what it was.

-- Vikas

Another note on this issue: I was working from a database to XML, and was confused by the fact that when a schema is generated by from an existing database it is called schema.xml rather than xxx-schema.xml. It took me ages to work out that i had to rename to schema.xml to xxx-schema.xml, but after I had the .java files were generated correctly. Maybe the solution to this confusion is to have turbine issue a helpful warning message when it finds schema.xml but not xxx-schema.xml?

-- StuartYeates

tbd

Hope this helps

--pete

Pete's tip is the solution when you get "Exception thrown by 'generator.parse'" errors from ant.

When I got errors about @DATABASE_DEFAULT@ from ant create-db, I found that changing that string in id-table-schema.xml to the name the database listed in my schema made things run smoother. Notably fixing "Unknown database" errors from create-db.

--ls

This is somewhat related to the problem encountered by Pete. Received a 'generator.parse' error using the default target from Ant.

The problem turned out to be from a datatype in the source Oracle schema. Used the torque 'jdbc' task and the resulting xxx-schema.xml file had an empty type attribute for any columns which were defined as Timestamps in the source Oracle schema. Modified the original ddl to change all Oracle 'Timestamp' columns over to the type 'Date'. Once I rebuilt the schema in Oracle, re-ran the torque jdbc task, torque's default (main) task worked fine.

-- Kyong Park (2004-04-27)

I also recieved an "Exception thrown by 'generator.parse'" error when using the Torque Maven plugin. This was fixed by specifying torque.useClasspath = false in my properties file.

-- Pål Brattberg (2004-06-06)

I'm using postgresql 8.0. Getting table definitions only worked for me after adding 'torque.database.schema = %' to the build.properties file.

-- seb

When using PostgreSQL the jdbc-target creates a schema.xml-File which reads "<database name="postgres">" - this is the user that connected to the database and therefore wrong. To have working java-Files generated later on, this entry needs to be changed manually to your database name.

-- StefanGaffga

another hint

If you get an error like:

[torque-sql] Couldn't read DTD specified in XML schema: db.apache.org [torque-sql] java.net.MalformedURLException

check in your conf files the Doctypes. <!DOCTYPE database SYSTEM "http://db.apache.org/torque/dtd/database_3_1.dtd">

-- knex

MS SQL Server insert-sql errors

If you're using the Microsoft SQL drivers, you must make sure all 3 .JAR files have their .jar extension in lowercase before putting them into the /lib folder. When you install the drivers, 2 of them are in uppercase by default. ANT only includes lowercase .jar files, leading to a ClassDefNotFound Exception when running the insert-sql task.

-- BarnyRelph

Torque tells you "IdGenerator for table 'xxx' is null"

After creating a Database-Scheme and successfully compiling the sources and creating the tables, i got the mentioned Error-Message. I spent a lot of time to find out what went wrong. The Sequences were created (i'm using an oracle Database) and everything seems to be fine. I put the word "native" in every place where it could make sense to force the engine to use my sequences but always the same Exception appeared.

'Solution:' You have to set the "adapter"-value for you Database-Connection. Otherwise the Engine does not know how to receive the PK. (If you use IDBROKER-Method in your scheme, you do not need this "adapter"-entry, but how to know, that you NEED this entry, if you do not use the ID-Table :-/ )

torque.database.default=survey_db
torque.database.survey_db.adapter=oracle

-- SyncMaster

This error may also occur if Torque is initialized twice for any reason. The solution for this is to ask Torque if it is already initialized via Torque.isInit() before initializing Torque.

-- Thomas Fischer

Also check to see that you in fact are using a valid value for idMethod in your schema.xml file. One example is using 'idBroker' instead of 'idbroker'.

-- Thierry Lach

Getting a Torque application to run from the command line

This is a listing of the typical problems people have trying to get their first Torque Application to run.

When you're using ant 1.4.1 you'll have this error when trying to execute jdbc ant task.

 Error "Could not create task of type: torque-jdbc-transform" 

 ant -f build-torque.xml jdbc 
 Buildfile: build-torque.xml 

 jdbc: 
    ["echo"] +-----------------------------------------------+ 
    ["echo"] |                                               | 
    ["echo"] | Generating XML from JDBC connection !         | 
    ["echo"] | Woo hoo!                                      | 
    ["echo"] |                                               | 
    ["echo"] +-----------------------------------------------+ 

 BUILD FAILED 

 ..../build-torque.xml:351: Could not create task of type: torque-jdbc-transform 
 due to java.lang.VerifyError: (class: org/apache/torque/task/TorqueJDBCTransformTask, method: generateXML signature: ()V) 
 Incompatible object argument for function call 

 Total time: 1 second

Typical eror when using torque with Ant 1.4.1. You need Ant 1.5.1 to use torque.

tbd

I am currently using Ant 1.6.1 and am getting this error for the custom task. Soft option is to comment out the task declaration in the build-torque.xml file, but I haven't yet determined the ultimate effects of this (java and sql generates OK and create-db target runs OK so far). Will post fix here if I find it.

-- JW (2004-07-02).

Getting Torque to run within a web-server

This section deals with the trials and tribulations of getting your web-applications Torque-enabled.

There are several steps to help check that Torque is setup correctly with your web-application, before you start testing Criteria, Peers, etc.

  1. Make sure that you can run the Torque sample applications from the command line. MANY PEOPLE FAIL THIS STEP.

    • If you fail this step, you need to review the item on getting Torque to run on the command line. (Usually, there's a problem in the Torque.properties file, or libraries are missing, or something.)
  2. Write a very simple JSP file that contains a small scriplet that prints the results of "Torque.isInit()" Make sure it's true.
    • If this step fails, you need to check a number of things:
      • Make sure you are initializing Torque - many people make a TorqueInitServlet and call Torque.init() when the Servlet starts.

      • Make sure the webserver can find the Torque.properties file. Usually this means specify an absolute path to the file Because of the way many webservers start up, you can't guarantee what the starting directory is for the webserver. Thus, local paths to the file often fail.

      • Make sure that you've included your database driver in the WEB-INF/lib directory.
      • Make sure that you've included the appropriate Torque libraries in the WEB-INF/lib directory.

Getting Torque to run within a Junit test

This section deals with the trials and tribulations of getting your unit tests to work with Torque.

I have a base TorqueTestCase that I extend all of my test cases that use torque. I use this for any junit based tests I run. Although I have found that running Cactus style in-container tests works better for me over the long run where the container starts Torque up, but this will help you write tests if you are not in a webapp:

Create a class looking like this:

   public class TorqueTestCase extends TestCase { 

   public final static String TORQUE_PROPS = new String( "c:/fortius/src/test/conf/Torque.properties" ); 
   private static Category log = Category.getInstance( BaseTestCase.class.getName() ); 
  
  /** 
   *  Constructor for the AntTaskRunnerTest object 
   * 
   *@param  name  Description of Parameter 
   */ 
  public TorqueTestCase ( String name ) { 
    super( name ); 

    if ( !Torque.isInit() ) { 
      log.debug( "Now starting Torque with path:" + TORQUE_PROPS ); 
      try { 
        Torque.init( TORQUE_PROPS ); 
      } 
      catch ( org.apache.torque.TorqueException te ) { 
        log.error( "Could not start Torque!" ); 
        fail( te.toString() ); 
      } 
    } 
  }

Now, admittedly the torque.props is hardcoded, but this should get you started. Just make sure all your constructors do call the super(name) method!

-- EricPugh

Using Torque with multiple databases

This section deals with the trials and tribulations of using multiple databases with Torque.

By 'multiple databases', I mean the use case where there are (conceivably) multiple distinct database engines, each of which hosts one or more databases, all or many of which have distinct schema. We wish to access multiple databases from this environment from within a particular JVM/classloader/class, during a single run of the program. This tortured description brought to you in hopes of being very clear on what this HOWTO is for. Thanks to B. Yu for pointing out a possibile ambiguity. [I have removed his note as I believe it could be confusing, no insult intended!]

It's fairly easy to get multiple databases working. The following examples assume you are using JNDI, since it's what I know best, but the differences are restricted to the torque.properties file. Start off by choosing a handle name for each of the databases to which you will connect. In this example there will be two databases, with handles dbA and dbB. Do the following:

  1. In the schema file for dbA, the <database> element must have a name attribute with value dbA. Likewise for the dbB schema file. When the object model files are generated (by ant -f build.properties om), the name will be hard-coded in them, and cannot be overriden.

  2. In the torque.properties file, create an adapter for each of your databases:

torque.database.dbA.adapter=mysql
torque.dsfactory.dbA.factory=org.apache.torque.dsfactory.JndiDataSourceFactory
torque.dsfactory.dbA.jndi.path=java:comp/env/jdbc/dbA

torque.database.dbB.adapter=mysql
torque.dsfactory.dbB.factory=org.apache.torque.dsfactory.JndiDataSourceFactory
torque.dsfactory.dbB.jndi.path=java:comp/env/jdbc/dbB

The key point here is the handle, emphasized in torque.database.dbA.adapter=mysql. The torque.properties file handles and the handles in the schema files must correspond. This is the a fundamental issue with getting multiple databases working with Torque.

  1. Every use of the database in the code must reference a handle [this is a simplification, there is a default handle, and rules for automatic creation of a default handle, but... let's not go there]. Here is an example:

  Criteria criteria = new Criteria("dbA");
  criteria.add(SomePeer.SOME_FIELD, "some value");
  SomePeer.doDelete(criteria);

-- Gary Shea

Installing from CVS

See: Building

Some further hints:

  1. When you install Maven, don't forget to set the environment variable MAVEN_HOME to the appropriate directory and to add the maven binary directory to the environment variable PATH.
  2. If/when Maven complains about not being able to download jdbc-2.0.jar and jndi-1.2.1.jar you will need to download these manually and install them in your local Maven repository due to licensing. jdbc-2.0 is available at http://java.sun.com/products/jdbc/download.html under the file name jdbc2_0-stdext.jar (click on the download link for JDBC 2.0 Optional Package Binary to get it). Download it, rename it to jdbc-2.0.jar and put it into the .maven/repository/jdbc/jars in your home directory. jndi-1_2_1.zip is available from http://java.sun.com/products/jndi/downloads/ (click the download link for Download JNDI 1.2.1 & More to get to it) - copy the file lib/jndi.jar into the directory .maven/repository/jndi/jars in your home diretory and rename it to jndi-1.2.1.jar. Do same for other jars.

-- Thomas Fischer (with major edits by ScottEade)

Using Torque with an existing database schema

  1. Verify that the JDBC driver and connection string is working - this can be easily done using SQL Squirrel since it is a pure JDBC client. If you can connect to your database, browse through your tables and view the JDBC metadata everything is fine. BTW SQL Squirrel also shows the JDBC class being used (saves you some digging through the docs)
  2. Set the Torque.properties with settings from 1. and run the 'jdbc' target to create a schema.xml
  3. It is time to look at the schema.xml since Torque might be confused about unknown column types and default values using proprietary SQL functions.
  4. A good idea is to consider using complex object types by setting <database defaultJavaType="object">. This allows NULLable values within your Torque generated classes.

  5. Now it is time to run the 'om' and 'compile' target with the update schema.xml to generate the database layer and check if it really compiles
  6. As a final step you can create HTML documentation using the 'doc' target

-- Siegfried Goeschl


Maven, Torque and the torque:jdbc goal

If have spent some time trying to get maven to work with torque:jdbc and there seems to be little doco on either (at least for a complete maven and torque novice). So here are the steps I used to get it working with Microsoft SQLServer.

  1. Install an appropriate Java SDK
  2. Add the Java SDK bin directory to your path variable so that you can type 'java' at the command prompt and it successfully starts.
  3. Install Maven
  4. Add the Maven bin directory to your path variable so that you can type 'maven' at the command prompt and it successfully starts.
  5. Use Maven to install the torque plugin as follows: (check the [Maven-howto for the latest details on this|http://db.apache.org/torque/maven-howto.html])

  6. Create yourself a project directory (this can be anywhere you like).
  7. In your project directory create a project.properties file as follows:

# -------------------------------------------------------------------
#
#  T O R Q U E   S E T T I N G S
#
# -------------------------------------------------------------------

torque.project = MyDatabase
torque.database = mssql

torque.database.createUrl = jdbc:microsoft:sqlserver://localhost:1433
torque.database.buildUrl = jdbc:microsoft:sqlserver://localhost:1433;DatabaseName=MyDatabase
torque.database.url = jdbc:microsoft:sqlserver://localhost:1433;DatabaseName=MyDatabase
torque.database.driver = com.microsoft.jdbc.sqlserver.SQLServerDriver
torque.database.user = dbusername
torque.database.password = dbpassword
torque.database.host = localhost
torque.database.schema = %

Note the schema setting may be different for different database types as a hint the following java call is made to return tables

tableNames = conn.getMetaData().getTables(null, schema, "%", types);

Where schema is taken from the torque.database.schema settings in the project.properties files. If when you get to the generation step and it appears to works but you have a schema.xml file which contains no tables then its probably because the schema setting is wrong.

  1. create a src directory under your project directory
  2. create a schema directory under the src directory you just created.
  3. Obtain the jars for your chosen databases' jdbc library. e.g. for SQL Server you need msutil.jar, msbase.jar and mssqlserver.jar
  4. I had a real problem injecting these jars into the classpath, all of the normal techniques don't work with maven. I have found some doc which says to add the jars to the torque project.xml dependancies but I was unable to get this method to work and as soon as you update (re-install) the torque plugin you loose your dependancies. So the hack I came up with is to copy the database jdbc jars into the maven install. Under the maven installation directory (as of maven 1.0.2) you will find a lib\endorsed directory. Just copy your database jars into there and all will be well.
  5. run maven torque:jdbc at the command prompt making sure that your current working directory is your project directory.
  6. You should seem something similar to :

C:\dev\torque>maven torque:jdbc
 __  __
|  \/  |__ _Apache__ ___
| |\/| / _` \ V / -_) ' \  ~ intelligent projects ~
|_|  |_\__,_|\_/\___|_||_|  v. 1.0.2

build:start:

torque:init:

torque:jdbc:
    [torque-jdbc-transform] Torque - JDBCToXMLSchema starting
    [torque-jdbc-transform] Your DB settings are:
    [torque-jdbc-transform] driver : com.microsoft.jdbc.sqlserver.SQLServerDriver
    [torque-jdbc-transform] URL : jdbc:microsoft:sqlserver://localhost:1433;DatabaseName=MyDatabase
    [torque-jdbc-transform] user : dbusername
    [torque-jdbc-transform] schema : %
    [torque-jdbc-transform] DB driver sucessfuly instantiated
    [torque-jdbc-transform] DB connection established
    [torque-jdbc-transform] Getting table list...
    [torque-jdbc-transform] Building column/table map...
    [torque-jdbc-transform] Processing table: dtproperties

... you should see each table and view in your database listed here ...

    [torque-jdbc-transform] Processing table: tblAccount
    [torque-jdbc-transform] Processing table: tblAllocation
    [torque-jdbc-transform] Processing table: vwWatchList
    [torque-jdbc-transform] C:\dev\torque/src/schema/schema.xml
    [torque-jdbc-transform] Torque - JDBCToXMLSchema finished
BUILD SUCCESSFUL
Total time: 16 seconds
Finished at: Sat Feb 05 06:47:00 EST 2005
  1. At the end of the above process you torque should have created a schema.xml under your project directory src/schema/
  2. Check the schema.xml actually contains your databases tables. If not check the torque.database.schema setting in your project.properties file.
  3. Rename the schema.xml to MyDatabase-schema.xml where MyDatabase is the value you entered in your project.properties file 'torque.project = '. This is required as all other torque goals (e.g. maven torque:datadump) require the schema.xml to follow the above naming convention (how dumb is that!).

Hope this helps.

BrettSutton

Errors during compile of Base objects

If you have database columns with names that are java key words your compile will fail. Torque does not check for this in v3.1.1 and it will attempt to make java classes with properties names like  class abstract public private . See http://java.sun.com/docs/books/tutorial/java/nutsandbolts/_keywords.html for a list of keywords. Those keywords are some rather seductive names for columns. Your options are to change your database column names from something like class to someprefix_class or to use the torque's javaName attribute in your schema.xml {{{<table name="classes">

Using ant to build 3.1.1

Here are some hints for UsingAntForV311


a great hands on tutorial at http://users.volja.net/amanita1/

PROBLEM: jdbc (generate XML schema) task runs to completion, no errors but no tables are in the generated schema.xml ENVIRONMENT: hsqlDB 1.7.2 and Torque GEN 3.2-rc1

Sample trace when problem exists:

[torque-jdbc-transform] Torque - JDBCToXMLSchema starting [torque-jdbc-transform] Your DB settings are: [torque-jdbc-transform] driver : org.hsqldb.jdbcDriver [torque-jdbc-transform] URL : jdbc:hsqldb:hsql://localhost:1701/ [torque-jdbc-transform] user : sa [torque-jdbc-transform] schema : ${torque.database.schema} [torque-jdbc-transform] DB driver sucessfuly instantiated

SOLUTION:

The clue is the schema name. HSQLDB 1.7.2 and earlier apparently does not support schemas and the property file delivered with Torque has torque.database.schema commented out. I guess Torque finds no tables in some unknown schema and exits.

Place empty assigment for the schema name into build.properties as such:

torque.database.schema =

That works!

Creating tables for an MS SQL database

If you try to use the command 'maven torque:create-db' to generate your database and tables using MS SQL, you will get an error saying:


[torque-data-model] Using classpath [torque-data-model] Generating to file D:\Downloads\mytorquetest\target\sql\create-db.sql [echo]

[sql] Executing file: D:\Downloads\mytorquetest\target\sql\create-db.sql [sql] [ERROR] Failed to execute: ECHO Not implemented [sql] [ERROR] java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for JDBC][SQLServer]Incorrect syntax near the keyword 'Not'. [sql] 0 of 1 SQL statements executed successfully


Don't worry if you get this error, the way around it is to manually create the database and then use the command 'maven torque:insert-sql' to create your tables


Received exception inserting new object - "inserting null value in not-null primary key column"

When I tried to create my first Torque project (other than the tutorial), I got a really wierd error trying to save a new Biller object. Torque complained that it was trying to insert a null "biller id" (the primary key for the object) when it ran the insert.

The problem was that I had copied some of the XML from the tutorial (bookstore) and in particular had the xml that read "<database name='bookstore'..." at the top. Since my database name was actually "billers" (and was referenced like that in my Torque.properties), it caused the problem. Once I changed the name of the database to be the correct one, it went away.

-PeterHamlen

GettingTorqueToWork (last edited 2009-09-20 23:04:10 by localhost)