Summary
Reliability Tests Suite contains 100 tests including 13 that are considered as invalid currently. This suite is critical for testing Harmony readiness for running long large workloads such as Eclipse in the development mode. {*}
How to run reliability tests
Detailed instruction placed at http://harmony.apache.org/subcomponents/buildtest/index.html
- There is approximate list of commands that should be executed:
cd <new folder for bti> svn co https://svn.apache.org/repos/asf/harmony/enhanced/buildtest/trunk/infra cd infra ant -Dtest.suites="classlib,drlvm,reliability" install ant setup ant run
Results could be observed at build/results/reliability folder
2. In case of testing virtual machine in different location the following commands can help:
cd <new folder for bti> svn co https://svn.apache.org/repos/asf/harmony/enhanced/buildtest/trunk/infra cd infra
Modify file "framework.local.properties" in order to set the path to tested vm like:
reliability.parameters.required.jvm.under.test=C:\harmony-jdk-628209\jre\bin\java.exe
Start test running
ant -Dtest.suites=reliability install ant setup ant run
3. Here is also sample of individual test run if needed:
<vm folder>\working_vm\build\windows_x86_msvc_release\deploy\jdk\jre\bin\java.exe -classpath <bti folder>\build\checkouts\reliability\deploy\classes org.apache.harmony.test.reliability.share.ReliabilityTestStarter -testReliabilityMode longrunning -testTimeOut 1200 -testClassName org.apache.harmony.test.reliability.api.kernel.hooks.AddRmShtdwnHooksTest <vm folder>\working_vm\build\windows_x86_msvc_release\deploy\jdk\jre\bin\java.exe <bti folder>\build\checkouts\reliability\deploy\classes
Test run status
_Pass rates according to Automated Testing results:_
Platform |
Valid |
Failed |
Excluded |
Failed+excluded |
Pass rate |
Windows x86 |
87 |
2 |
13 |
15 |
83% |
Linux x86 |
87 |
2 |
13 |
15 |
83% |
Linux x86_64 |
87 |
4 |
17 |
21 |
76% |
Excluded test list with references to corresponding JIRA could be found in files:
build/checkouts/reliability/run/exclude build/checkouts/reliability/run/exclude.<platform>