Running the Apache Derby test suite on phoneME Advanced

Since phoneME b96 (r17519) it is possible to run the Derby testsuite (....suites.All) on the phoneME Advanced(aka cdc) virtual machine (cvm) for Linux X86 without having to locally modify the phoneME source or build script.

For general description of how to build phoneME Advanced see http://wiki.java.net/bin/view/Mobileandembedded/PhoneMEAdvancedBuildInstructions .

The following describes how to build a variant of phoneME Advanced that is able to run the Derby testsuite (....suites.All).

Download phoneME Advanced

Create a directory to hold the phoneME Advanced source:

  PHONE_ME_SRC=${SOME_WHERE}/phoneme-advanced # your choice...
  mkdir -p ${PHONE_ME_SRC}
  cd ${PHONE_ME_SRC} 

Check out the sources from http://java.net/projects/phoneme/sources:

  svn checkout https://svn.java.net/svn/phoneme~svn/components/tools/trunk tools
  svn checkout https://svn.java.net/svn/phoneme~svn/components/cdc/trunk cdc
  svn checkout https://svn.java.net/svn/phoneme~svn/legal legal

The following is not required for doing the build.

  svn checkout https://svn.java.net/svn/phoneme~svn/trunk phoneme

Build phoneME Advanced

  cd ${PHONE_ME_SRC}/cdc/build/linux-x86-generic/

The build option 'LINUX_ENABLE_SET_AFFINITY=true' was introduced to force cvm onto one CPU. This is only needed if you want to run on a multi-cpu machine. Without this we have experienced hangs when running phoneME Advanced / cvm on a SMP machine.

  SETAFFINITY="LINUX_ENABLE_SET_AFFINITY=true" 

Options to use (this will not build cvm to include jsr169):

  SDK14_HOME=/usr/local/java/j2sdk1.4.2_13
  MK_WITHOUT_JSR169="JDK_HOME=${SDK14_HOME}/ CVM_JAVA_TOOLS_PREFIX=${SDK14_HOME}/bin/ CVM_JIT=false CVM_MP_SAFE=false J2ME_CLASSLIB=foundation ${SETAFFINITY}" 

In this case you will have to run cvm with "-Xbootclasspath/a:..../jsr169.jar" for a pre-built jsr169.

Build plain phoneME Advanced

Then build without including jsr169:

  echo ${MK_WITHOUT_JSR169}
  make ${MK_WITHOUT_JSR169} clean
  make ${MK_WITHOUT_JSR169}
  make ${MK_WITHOUT_JSR169} bin

The cvm virtual machine is then located in

  CVM=${PHONE_ME_SRC}/cdc/build/linux-x86-generic/bin/cvm

Build phoneME with jsr169 included

Alternatively, if you want to build with jsr169 included, add this:

  JSR169=${SOME_WHERE}/jsr169 # Where you have downloaded JSR169...
  MK_WITH_JSR169="${MK_WITHOUT_JSR169} USE_JDBC=true JDBC_DIR=${JSR169}/jdbc_cdc1.0/cdcfoundation"

This assumes you have downloaded and built jsr169 to ${JSR169}/. See A.1. for how to prepare jsr169.jar. In this case you do not need to use "-Xbootclasspath/a:..../jsr169.jar" when running cvm.

and then build with jsr169 included in cvm: (You might want to keep the "clean" cvm: 'mv ${CVM} <somewhere not beeing cleaned...>/cvm-no-jsr169') 

  echo ${MK_WITH_JSR169}
  make ${MK_WITH_JSR169} clean
  make ${MK_WITH_JSR169}
  make ${MK_WITH_JSR169} bin

The cvm virtual machine is still located in

  CVM=${PHONE_ME_SRC}/cdc/build/linux-x86-generic/bin/cvm

Test the build

Show which options was used when building cvm:

  ${CVM} -XshowBuildOptions

Show cvm info:

  ${CVM} -version

Observe that to run Derby on phoneME Advanced / cvm, Derby must be built with Foundation Profile 1.1. See A.2.

Test Derby on the build

For initial testing of Derby on cvm do: Specify where your Derby jars are located, e.g.

  DERBY_JARS=${SOME_WHERE}/Apache/Derby/trunk/jars/insane

or

  DERBY_JARS=/usr/local/share/java/derby/Derby-10.5.1.1/lib

Show Derby system info:

  ${CVM} -jar ${DERBY_JARS}/derbyrun.jar sysinfo 

Without jsr169 built in you will see: "... java.lang.NoClassDefFoundError: java.sql.Timestamp ..." In that case you will need to do

  JSR169_CLASSES=${SOME_WHERE}/jsr169/classes # Where you have jsr169 ....
  JDBCOp=jsr169
  ${CVM} -Xbootclasspath/a:${JSR169_CLASSES}/${JDBCOp}.jar -jar ${DERBY_JARS}/derbyrun.jar sysinfo

Run the Apache Derby test suite on the build

You should now be ready to run the Derby test suite(s) on cvm: If you are running without jsr169 built into cvm set

  BOOTCP="-Xbootclasspath/a:${JSR169_CLASSES}/${JDBCOp}.jar"

Specify the test you want to run:

  suite=org.apache.derbyTesting.functionTests.suites.All 

To run this large test suite you need to use

  UNLIMITEDGCROOTS="-XunlimitedGCRoots"
  CVMMEM="-Xgc:youngGen=5M -Xmn256M" 

If you want to trace the progress of the test set

  TRACEJUNIT="-Dderby.tests.trace=true" 

For the upgrade tests specify where your older version Derby jars are located, eg.:

  FOR_UPGRADE_TESTS="-DderbyTesting.oldReleasePath=/usr/local/share/java/derby/lib" 

And last but not least set your CLASSPATH for Derby:

  CLASSPATH=${DERBY_JARS}/derby.jar:${DERBY_JARS}/derbyclient.jar:${DERBY_JARS}/....  

Specify the command with all options:

  CMD="${CVM} ${BOOTCP} ${CVMMEM} ${UNLIMITEDGCROOTS} ${FOR_UPGRADE_TESTS} ${TRACEJUNIT} -cp ${CLASSPATH} junit.textui.TestRunner"

and start the test:

  ${CMD} ${suite} 


A.1. - Building jsr169

See http://wiki.java.net/bin/view/Mobileandembedded/PhoneMEAdvancedBuildingJDBCandRMI for the general description.

Set

  JSR169=${SOME_WHERE}/jsr169
  mkdir -p ${JSR169}
  JDBC_Source=${JSR169}/jdbc_cdc1.0

Download jdbc_cdc-1_0-rr-src.zip from https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jdbc_cdc_fp-1.0-rc-spec-oth-JSpec@CDS-CDS_Developer by following links from http://wiki.java.net/bin/view/Mobileandembedded/PhoneMEAdvancedBuildingJDBCandRMI (As of 2009-04-29).

Unpack jdbc_cdc-1_0-rr-src.zip to  ${JSR169}  which creates  ${JSR169}/jdbc_cdc1.0.

Assuming PhoneME Avanced advanced has been built without jsr169, see above, giving

  PHONE_ME_SRC=${SOME_WHERE}/phoneme-advanced

you should have

  ${PHONE_ME_SRC}/cdc/build/linux-x86-generic/lib/foundation.jar
  ${PHONE_ME_SRC}/cdc/build/linux-x86-generic/btclasses.zip

Set

  CDC_HOME=${PHONE_ME_SRC}/cdc/build/linux-x86-generic

You need to build with jdk1.4, so do something like:

  JDK_HOME=/usr/local/java/j2sdk1.4.2_13

Specify where the jsr169 classes will be put:

  JSR169_CLASSES=${JSR169}/classes
  mkdir -p ${JSR169_CLASSES} 

Do the build from java and javax

  cd ${JDBC_Source}/cdcfoundation/src/share/jdbc/classes
  # Contains java/ and javax/
  ${JDK_HOME}/bin/javac -bootclasspath ${CDC_HOME}/btclasses.zip:${CDC_HOME}/lib/foundation.jar \
    -d ${JSR169_CLASSES} java/sql/*.java
  ${JDK_HOME}/bin/javac -bootclasspath ${CDC_HOME}/btclasses.zip:${CDC_HOME}/lib/foundation.jar \
    -d  ${JSR169_CLASSES} javax/sql/*.java

Create the jsr169.jar:

  cd ${JSR169_CLASSES}
  JDBCOp=jsr169
  ${JDK_HOME}/bin/jar cvf ${JDBCOp}.jar java/sql/*.class javax/sql/*.class

${JDBCOp}.jar (${JSR169_CLASSES}/jsr169.jar) can now be used to build PhoneME Advanced to include jsr169. See above.

Or, if you did not have jsr169.jar available from some other source, you will need to use it running cvm with "-Xbootclasspath/a:..../jsr169.jar".


A.2. - Building Derby for jsr169

General building instructions for Derby are found in How to build Derby and BUILDING.html.

Your ant.properties file must contain something like:

jsr169compile.classpath=${SOMEWHERE}/btclasses.zip;${SOMEWHERE}/foundation.jar;${SOMEWHERE}/jsr169.jar

btclasses.zip and foundation.jar can be downloaded via http://java.sun.com/javame/index.jsp: (or you can use btclasses.zip}} and {{{foundation.jar created above.)

Follow 'Downloads' - http://java.sun.com/javame/downloads/index.jsp

Click 'Download' of "Foundation Profile 1.1 Reference Implementation (JSR 219) Foundation Profile RI with Security Optional Package Documentation" which (per 2008-12-29) brings you to

https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_JCP-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=fp-1.1-fr-ri-oth-JPR@CDS-CDS_JCP

Click 'Continue'.

Select and download (to an empty directory e.g. ${HOME}/FP1.1): 'Foundation Profile RI' 'fp-1_1-fr-ri.zip'

'cd ${HOME}/FP1.1'
'unzip fp-1_1-fr-ri.zip'

The file 'copyright.txt' and the directory 'Optimized_nosym/' are extracted. The 'btclasses.zip' and'foundation.jar' file which must be used for compiling for Foundation Profile 1.1 are then available in 'Optimized_nosym/lib/'.

(The phoneME / Java ME / Foundation Profile 1.1 Java Runtime 'cvm' is found in 'Optimized_nosym/bin/'.)

Alternatively you can use "your own" ${CDC_HOME}/btclasses.zip and ${CDC_HOME}/lib/foundation.jar from A.1. For jdbc.jar aka jsr169.jar you can use "your own" ${JDBCOp}.jar (${JSR169_CLASSES}/jsr169.jar) from A.1.


A.3. - phoneME Advanced / cvm platforms tested

Building phoneME Advanced / cvm and testing Derby with the build has been done on (at least) the following platforms:

Building:

Running the Apache Derby test suite on

RunningTheApacheDerbyTestSuiteOnPhoneMeAdvanced (last edited 2011-03-30 15:48:29 by KnutAndersHatlen)