Differences between revisions 20 and 21
Revision 20 as of 2006-05-26 20:16:27
Size: 7508
Comment:
Revision 21 as of 2009-09-20 21:54:55
Size: 7588
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
Eclipse is freely extensible by means of plugins. The list of plugins can be found at [http://www.eclipseplugincentral.com/ Eclipse Plugin Central]. There is testing plugins [http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-16.html subcategory] which includes
[http://www.eclipse.org/tptp/ Test and Performance Tools Platform] (TPTP) plugin. This plugin is an Eclipse top-level project which tries to establish a
Eclipse is freely extensible by means of plugins. The list of plugins can be found at [[http://www.eclipseplugincentral.com/|Eclipse Plugin Central]]. There is testing plugins [[http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-16.html|subcategory]] which includes
[[http://www.eclipse.org/tptp/|Test and Performance Tools Platform]] (TPTP) plugin. This plugin is an Eclipse top-level project which tries to establish a
Line 14: Line 14:
   * [http://www.junit.org/ JUnit] is a recommended tool for developing tests. Is de-facto standard and it is integrated into Eclipse by default.
   * [http://testng.org/ TestNG] is a new generation of JUnit tests which is integrated into Eclipse by means of the following [http://testng.org/doc/eclipse.html plugin]. It can run JUnit tests without any changes, and has an automatic converter for JUnit tests to testNG format. Javadoc taglets or java 5.0 annotations can be used to specify test types and dependencies.
   * [[http://www.junit.org/|JUnit]] is a recommended tool for developing tests. Is de-facto standard and it is integrated into Eclipse by default.
   * [[http://testng.org/|TestNG]] is a new generation of JUnit tests which is integrated into Eclipse by means of the following [[http://testng.org/doc/eclipse.html|plugin]]. It can run JUnit tests without any changes, and has an automatic converter for JUnit tests to testNG format. Javadoc taglets or java 5.0 annotations can be used to specify test types and dependencies.
Line 23: Line 23:
   * Eclipse has a primitive test finder which allows to [http://open.ncsu.edu/se/tutorials/junit/#section4_0 run] all tests in the selected project, package or source folder.    * Eclipse has a primitive test finder which allows to [[http://open.ncsu.edu/se/tutorials/junit/#section4_0|run]] all tests in the selected project, package or source folder.
Line 25: Line 25:
   * There is a [https://bugs.eclipse.org/bugs/show_bug.cgi?id=10462 feature request] for advanced test finders.    * There is a [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=10462|feature request]] for advanced test finders.
Line 38: Line 38:
[http://www.eclipse-plugins.info/eclipse/plugin_details.jsp?id=668 Release Engineering plugin] used by Eclipse team is [http://www.eclipsecon.org/2006/Sub.do?id=202 reported] to be hardly configurable. Though there are many aternatives. [[http://www.eclipse-plugins.info/eclipse/plugin_details.jsp?id=668|Release Engineering plugin]] used by Eclipse team is [[http://www.eclipsecon.org/2006/Sub.do?id=202|reported]] to be hardly configurable. Though there are many aternatives.
Line 42: Line 42:
CVS is a default source control system for Eclipse. Many other source control servers such as Subversion are CVS-compatible. Specialized plugins can be found [http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-15.html here]. CVS is a default source control system for Eclipse. Many other source control servers such as Subversion are CVS-compatible. Specialized plugins can be found [[http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-15.html|here]].
Line 52: Line 52:
   * By default the internal Eclipse builder, [http://ant.apache.org Apache Ant] and a command line tool are supported. Eclipse is extensible with custom builders.
   * Eclipse is integrated with Maven by means of the following[http://maven.apache.org/plugins/maven-eclipse-plugin/overview.html plugin].
   * Eclipse requires [http://www.eclipse.org/cdt/ C/C++ development tools] to be installed to edit [http://www.gnu.org/software/make/ GNU makefiles].
   * By default the internal Eclipse builder, [[http://ant.apache.org|Apache Ant]] and a command line tool are supported. Eclipse is extensible with custom builders.
   * Eclipse is integrated with Maven by means of the following[[http://maven.apache.org/plugins/maven-eclipse-plugin/overview.html|plugin]].
   * Eclipse requires [[http://www.eclipse.org/cdt/|C/C++ development tools]] to be installed to edit [[http://www.gnu.org/software/make/|GNU makefiles]].
Line 58: Line 58:
[http://www.martinfowler.com/articles/continuousIntegration.html Continuous Integration] is an [http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=35638 agile] software development practice for a teamwork with the following features: [[http://www.martinfowler.com/articles/continuousIntegration.html|Continuous Integration]] is an [[http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=35638|agile]] software development practice for a teamwork with the following features:
Line 64: Line 64:
   * Here is a [http://www-128.ibm.com/developerworks/edu/os-dw-os-cruise-i.html tutorial] on integration of the popular coninuous integration systems [http://cruisecontrol.sourceforge.net/ CruiseControl], [http://sourceforge.net/projects/luntbuild Luntbuild], and [http://www.urbancode.com/projects/anthill/default.jsp Anthill].
   * [http://pag.csail.mit.edu/continuoustesting/ Continuous Testing] technology runs unit tests in parallel with development process.
   * TPTP has an extensible SOA-based [http://www.eclipsecon.org/2006/Sub.do?id=99 design].
   * [http://sourceforge.net/projects/cruise-eclipse/ CruiseController] Eclipse Plugin allows remote management of [http://cruisecontrol.sourceforge.net/ CruiseControl] build queues.
   * Here is a [[http://www-128.ibm.com/developerworks/edu/os-dw-os-cruise-i.html|tutorial]] on integration of the popular coninuous integration systems [[http://cruisecontrol.sourceforge.net/|CruiseControl]], [[http://sourceforge.net/projects/luntbuild|Luntbuild]], and [[http://www.urbancode.com/projects/anthill/default.jsp|Anthill]].
   * [[http://pag.csail.mit.edu/continuoustesting/|Continuous Testing]] technology runs unit tests in parallel with development process.
   * TPTP has an extensible SOA-based [[http://www.eclipsecon.org/2006/Sub.do?id=99|design]].
   * [[http://sourceforge.net/projects/cruise-eclipse/|CruiseController]] Eclipse Plugin allows remote management of [[http://cruisecontrol.sourceforge.net/|CruiseControl]] build queues.
Line 71: Line 71:
[http://abbot.sourceforge.net/ Abbot] and other [http://www.junit.org/news/extension/gui/index.htm JUnit based testing tools] are used to create automated tests for GUI applications. [http://www.eclipse.org/tptp/ Test and Performance Tools Platform] embeds these tools and enriching them with Eclipse-specific event hooks. [[http://abbot.sourceforge.net/|Abbot]] and other [[http://www.junit.org/news/extension/gui/index.htm|JUnit based testing tools]] are used to create automated tests for GUI applications. [[http://www.eclipse.org/tptp/|Test and Performance Tools Platform]] embeds these tools and enriching them with Eclipse-specific event hooks.
Line 75: Line 75:
   * TPTP describes [http://www.eclipse.org/tptp/home/documents/conferences/eclipseCon2006/EclipseCon_2006_Long_Talk_Paul_Slauenwhite_files/frame.htm remote agents] architecture.
   * JUnit has a [http://www.junit.org/news/extension/j2ee/index.htm set] of distributed test runners
   * TPTP describes [[http://www.eclipse.org/tptp/home/documents/conferences/eclipseCon2006/EclipseCon_2006_Long_Talk_Paul_Slauenwhite_files/frame.htm|remote agents]] architecture.
   * JUnit has a [[http://www.junit.org/news/extension/j2ee/index.htm|set]] of distributed test runners
Line 81: Line 81:
   * [http://www.eclipse.org/tptp/performance/documents/design/arch_tptp_traceprofile.html Tracing and Profiling Tools Project]    * [[http://www.eclipse.org/tptp/performance/documents/design/arch_tptp_traceprofile.html|Tracing and Profiling Tools Project]]
Line 85: Line 85:
Here is a [http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-14.html list] of Eclipse plugins which provide source code metrics. Here is a [[http://www.eclipseplugincentral.com/Web_Links-index-req-viewlink-cid-14.html|list]] of Eclipse plugins which provide source code metrics.
Line 90: Line 90:
   * [http://clarkware.com/software/JUnitPerf.html JUnitPerf] combines tests programmatically.
   * [http://junitpp.sourceforge.net/ JUNITPP] allows combining tests from the command line.
   * [[http://clarkware.com/software/JUnitPerf.html|JUnitPerf]] combines tests programmatically.
   * [[http://junitpp.sourceforge.net/|JUNITPP]] allows combining tests from the command line.
Line 95: Line 95:
Most [http://www.junit.org/news/extension/testcase_generation/index.htm test generators] just help test developers to obtain a trivial JUnit template. The folloiwng generators are more complex and worth additional investigation. Most [[http://www.junit.org/news/extension/testcase_generation/index.htm|test generators]] just help test developers to obtain a trivial JUnit template. The folloiwng generators are more complex and worth additional investigation.
Line 97: Line 97:
   * [http://stud4.tuwien.ac.at/~e9825234/javver/index.html Javver] generates random bytecode
   * [http://cricketcage.sourceforge.net/ Cricket Cage] creates test cases recording an exeecution of a valid program
   * [[http://stud4.tuwien.ac.at/~e9825234/javver/index.html|Javver]] generates random bytecode
   * [[http://cricketcage.sourceforge.net/|Cricket Cage]] creates test cases recording an exeecution of a valid program
Line 103: Line 103:
By default Eclipse reports JUnit results in a widget. [http://beust.com/testng/test-output/index.html TestNG] have its own HTML reporting. JUnit has an [http://sourceforge.net/projects/junitpdfreport extension] to geneerate PDF reports. By default Eclipse reports JUnit results in a widget. [[http://beust.com/testng/test-output/index.html|TestNG]] have its own HTML reporting. JUnit has an [[http://sourceforge.net/projects/junitpdfreport|extension]] to geneerate PDF reports.
Line 109: Line 109:
 1. [http://www.eclipse.org/tptp/home/documents/conferences/eclipseWorld2005/EclipseWorld_2005_Testing_Tools_Inside_Eclipse_new.htm Testing Tools Inside Eclipse]  1. [[http://www.eclipse.org/tptp/home/documents/conferences/eclipseWorld2005/EclipseWorld_2005_Testing_Tools_Inside_Eclipse_new.htm|Testing Tools Inside Eclipse]]

[Draft]

This page lists testing tools integrated with Eclipse IDE by topic. How this list is related to Harmony project? Since integrated development environments are collections of mature development tools, it worth to check this list before starting tool implementation from scratch.

Eclipse Extensibility Model

Eclipse is freely extensible by means of plugins. The list of plugins can be found at Eclipse Plugin Central. There is testing plugins subcategory which includes Test and Performance Tools Platform (TPTP) plugin. This plugin is an Eclipse top-level project which tries to establish a unified platform for testing tools.

Test Formats

  • JUnit is a recommended tool for developing tests. Is de-facto standard and it is integrated into Eclipse by default.

  • TestNG is a new generation of JUnit tests which is integrated into Eclipse by means of the following plugin. It can run JUnit tests without any changes, and has an automatic converter for JUnit tests to testNG format. Javadoc taglets or java 5.0 annotations can be used to specify test types and dependencies.

Test Execution

Test Finders

Test finders allow to select and run a subset of available test base.

  • Eclipse has a primitive test finder which allows to run all tests in the selected project, package or source folder.

  • A test can be excluded from test run by excluding the test from build process (<Right Mouse Button> -> Build Path -> Exclude).

  • There is a feature request for advanced test finders.

Release Engineering

Release Engineering is a process of creating binary builds from source code. It includes the following steps:

  • Acquire a source code from a source contol system
  • Build the source code
  • Bundle the binaries
  • Run promotion checks
  • Maintain integrity of the source repository

If promotion checks pass, the build is promoted for the test cycle.

Release Engineering plugin used by Eclipse team is reported to be hardly configurable. Though there are many aternatives.

Source Control

CVS is a default source control system for Eclipse. Many other source control servers such as Subversion are CVS-compatible. Specialized plugins can be found here.

Build Tools

Build tools are script engines which handle local and remote dependencies.

  • Local dependency management allows incremental builds which rebuild changed files only and files indirectly affected by the change.
  • Remote dependencies are taken into account for automatic download and updates of binary or source library distributions from the Internet.

Eclipse has the following support for building tools:

  • XML-based build scripts are highlighted and folded in the editor.
  • By default the internal Eclipse builder, Apache Ant and a command line tool are supported. Eclipse is extensible with custom builders.

  • Eclipse is integrated with Maven by means of the followingplugin.

  • Eclipse requires C/C++ development tools to be installed to edit GNU makefiles.

Continuous Integration

Continuous Integration is an agile software development practice for a teamwork with the following features:

  • Daily synchronization with a common repository
  • Continuous repository verification (automated build and testing)

Eclipse supports the following systems for continuous integration:

UI Testing Tools

Abbot and other JUnit based testing tools are used to create automated tests for GUI applications. Test and Performance Tools Platform embeds these tools and enriching them with Eclipse-specific event hooks.

Distributed Testing Tools

  • TPTP describes remote agents architecture.

  • JUnit has a set of distributed test runners

  • TestNG intself supports distribution of operations on slave machines

Tracing and Profiling Tools

Source Code Coverage and Other Metrics

Here is a list of Eclipse plugins which provide source code metrics.

Stress/Performance Test Generators

The followng tools allows combining existent functional JUnit tests into stress/performance tests.

  • JUnitPerf combines tests programmatically.

  • JUNITPP allows combining tests from the command line.

Automatic Test Generators

Most test generators just help test developers to obtain a trivial JUnit template. The folloiwng generators are more complex and worth additional investigation.

  • Javver generates random bytecode

  • Cricket Cage creates test cases recording an exeecution of a valid program

Test Reports

By default Eclipse reports JUnit results in a widget. TestNG have its own HTML reporting. JUnit has an extension to geneerate PDF reports.

Additional References

  1. Mikhail Voronin at Intel
  2. Testing Tools Inside Eclipse

Eclipse_Testing_Tools (last edited 2009-09-20 21:54:55 by localhost)