Differences between revisions 18 and 19
⇤ ← Revision 18 as of 2007-06-28 14:52:48 →
Size: 4497
Editor: TatyanaDoubtsova
Comment:
← Revision 19 as of 2009-09-20 21:55:35 → ⇥
Size: 4531
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
We have just started to create [http://people.apache.org/builds/harmony/milestones/ stable builds] of the recently tested code provided for your convenience. These builds are declared best-so-far, but they are not Apache Harmony project official releases. Here you can find the schedule and detailed goals for the M2 milestone. We have just started to create [[http://people.apache.org/builds/harmony/milestones/|stable builds]] of the recently tested code provided for your convenience. These builds are declared best-so-far, but they are not Apache Harmony project official releases. Here you can find the schedule and detailed goals for the M2 milestone.
Line 24: Line 24:
 * 98% pass rate for '''VM validation test suite''' - [http://issues.apache.org/jira/browse/HARMONY-3206 HARMONY-3206]; no regression since M1.
 * 98% pass rate for '''Functional test suite''' - [http://issues.apache.org/jira/browse/HARMONY-3528 HARMONY-3528]; no regression since M1. The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4107 HARMONY-4107].
 * 98% pass rate for '''Stress tests''' - [http://issues.apache.org/jira/browse/HARMONY-3536 HARMONY-3536]; no regression since M1. The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4108 HARMONY-4108].
 * 98% pass rate for '''Reliability test''' - [http://issues.apache.org/jira/browse/HARMONY-2918 HARMONY-2918]; no regression since M1. The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4110 HARMONY-4110].
 * 98% pass rate for '''Eclipse 3.2 unit tests'''; no regression since M1. The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4109 HARMONY-4109].
 * 98% pass rate for '''Geronimo unit tests'''; no regression since M1. The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4111 HARMONY-4111].		  * 98% pass rate for '''VM validation test suite''' - [[http://issues.apache.org/jira/browse/HARMONY-3206|HARMONY-3206]]; no regression since M1.
 * 98% pass rate for '''Functional test suite''' - [[http://issues.apache.org/jira/browse/HARMONY-3528|HARMONY-3528]]; no regression since M1. The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4107|HARMONY-4107]].
 * 98% pass rate for '''Stress tests''' - [[http://issues.apache.org/jira/browse/HARMONY-3536|HARMONY-3536]]; no regression since M1. The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4108|HARMONY-4108]].
 * 98% pass rate for '''Reliability test''' - [[http://issues.apache.org/jira/browse/HARMONY-2918|HARMONY-2918]]; no regression since M1. The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4110|HARMONY-4110]].
 * 98% pass rate for '''Eclipse 3.2 unit tests'''; no regression since M1. The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4109|HARMONY-4109]].
 * 98% pass rate for '''Geronimo unit tests'''; no regression since M1. The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4111|HARMONY-4111]].
Line 37: Line 37:
 * Run Jetty scenario (link). The goal is not met currently. The umbrella task is [http://issues.apache.org/jira/browse/HARMONY-4112 HARMONY-4112].  * Run Jetty scenario (link). The goal is not met currently. The umbrella task is [[http://issues.apache.org/jira/browse/HARMONY-4112|HARMONY-4112]].
Line 39: Line 39:
 * All [http://dacapobench.org Dacapo] benches correctly with all workloads; new features or patches committed prior M2 introduce no degradations.
 * [http://math.nist.gov/scimark2 Scimark] benchmark works correctly; new features or patches committed prior M2 introduce no degradations.
 * Hamrony usability should be equal or better than of regular Java implementations. There are two currently known bugs in this category: [http://issues.apache.org/jira/browse/HARMONY-3251 HARMONY-3251] and [http://issues.apache.org/jira/browse/HARMONY-4069 HARMONY-4069].		  * All [[http://dacapobench.org|Dacapo]] benches correctly with all workloads; new features or patches committed prior M2 introduce no degradations.
 * [[http://math.nist.gov/scimark2|Scimark]] benchmark works correctly; new features or patches committed prior M2 introduce no degradations.
 * Hamrony usability should be equal or better than of regular Java implementations. There are two currently known bugs in this category: [[http://issues.apache.org/jira/browse/HARMONY-3251|HARMONY-3251]] and [[http://issues.apache.org/jira/browse/HARMONY-4069|HARMONY-4069]].
Line 43: Line 43:
Results of automatic testing scenarios for [http://harmony.apache.org/downloads.html#Harmony5_Snapshot regular snapshots] are [http://harmony.apache.org/snapshots_v5.html here]. Results of automatic testing scenarios for [[http://harmony.apache.org/downloads.html#Harmony5_Snapshot|regular snapshots]] are [[http://harmony.apache.org/snapshots_v5.html|here]].

The current page stores key points of the M2 discussion.

Stable builds

We have just started to create stable builds of the recently tested code provided for your convenience. These builds are declared best-so-far, but they are not Apache Harmony project official releases. Here you can find the schedule and detailed goals for the M2 milestone.

M2 Schedule

We are planning to deliver stable builds once in two months. Below is the schedule for the M2 stable build:

  • 30-Jun-2007: M2 published

  • 23-Jun-2007: M2 code freeze. Critical bug fixes only.

  • 13-Jun-2007: M2 feature freeze. No major redesigns since that date.

M2 Goals

Each stable build should have best-achieved stability, performance, compatibility, and completeness. The goals below set specific focus for M2. Pass rates are measured for valid tests.

  • 98% pass rate for class library tests; no regression since M1.

  • 98% pass rate for drl vm tests; no regression since M1.

  • 98% pass rate for jdk_tools tests; no regression since M1.

  • 98% pass rate for VM validation test suite - HARMONY-3206; no regression since M1.

  • 98% pass rate for Functional test suite - HARMONY-3528; no regression since M1. The goal is not met currently. The umbrella task is HARMONY-4107.

  • 98% pass rate for Stress tests - HARMONY-3536; no regression since M1. The goal is not met currently. The umbrella task is HARMONY-4108.

  • 98% pass rate for Reliability test - HARMONY-2918; no regression since M1. The goal is not met currently. The umbrella task is HARMONY-4110.

  • 98% pass rate for Eclipse 3.2 unit tests; no regression since M1. The goal is not met currently. The umbrella task is HARMONY-4109.

  • 98% pass rate for Geronimo unit tests; no regression since M1. The goal is not met currently. The umbrella task is HARMONY-4111.

  • 98% pass rate for Eclipse TPTP profiler tests; no regression since M1.

  • Harmony works smoothly with Eclipse 3.2; no known bugs; no regression since M1.
  • Run Eclipse 3.2 Hello World Application scenario (link).
  • Run HDK self-hosting (link).
  • Run Simple Tomcat scenario (link).
  • Run Apache Axis scenario (link).
  • Run Apache Struts scenario (link).

  • Run Jetty scenario (link). The goal is not met currently. The umbrella task is HARMONY-4112.

  • 48 hour Eclipse scenario (link).

  • All Dacapo benches correctly with all workloads; new features or patches committed prior M2 introduce no degradations.

  • Scimark benchmark works correctly; new features or patches committed prior M2 introduce no degradations.

  • Hamrony usability should be equal or better than of regular Java implementations. There are two currently known bugs in this category: HARMONY-3251 and HARMONY-4069.

Results of automatic testing scenarios for regular snapshots are here.

There are results of snapshot r551077 testing:

Windows 32

Valid

Pass

Fail

Rate

Class Library tests

26230

25884

346

98.68%

DRL VM tests

99.32

jdk_tools tests

151

148

3

98.01%

Reliability tests

89

65

24

73.03%

Functional tests

1358

1309

49

96.39%

VM validation tests

4330

4275

55

98.73%

Stress tests

197

190

7

96.45%

Eclipse 3.2 unit tests

36801

99.32%

Geronimo unit tests

616

614

2

99.68%

Eclipse TPTP profiler tests

172

172

0

100%

The table is updated on Tuesday, June 28, 2007.

Feedback

milestones/M2 (last edited 2009-09-20 21:55:35 by localhost)