Differences between revisions 18 and 19
Revision 18 as of 2005-02-03 00:18:02
Size: 15288
Editor: ZacharySmith
Comment:
Revision 19 as of 2009-09-20 23:24:28
Size: 15264
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
Milton is a part test framework and utility and part methodology. It is comprised of a three parts. [[BR]] Milton is a part test framework and utility and part methodology. It is comprised of a three parts. <<BR>>
Line 6: Line 6:
 * Support for generating client code for excercising a Control. Using a ["Mantis"] processor the Milton generator will parse Controls test files, called Drivers and generate the corresponding JPF, JWS, and JUnit accessors allowing the test author to focus on writing the test logic and get end-to-end testing virtually for free.[[BR]]  * Support for generating client code for excercising a Control. Using a [[Mantis]] processor the Milton generator will parse Controls test files, called Drivers and generate the corresponding JPF, JWS, and JUnit accessors allowing the test author to focus on writing the test logic and get end-to-end testing virtually for free.<<BR>>
Line 19: Line 19:
 * a. User invokes JUnit Accessor method using either [wiki:/TCH TCH] or standard JUnit runner.  * a. User invokes JUnit Accessor method using either [[/TCH|TCH]] or standard JUnit runner.
Line 72: Line 72:
  * 4. The method must have the `@Milton.Test` annotation applied to it. This tells the Milton Client Generation processor that this is infact a Test method and that it must comply with the above conventions. There are four fields available on this annotation: `frequency()`, `status()`, `description()`, and `misc()`. These annotations are made available to work with [wiki:/TCH TCH]. Please see the [wiki:/TCH TCH] documentation for more information on their meanings.   * 4. The method must have the `@Milton.Test` annotation applied to it. This tells the Milton Client Generation processor that this is infact a Test method and that it must comply with the above conventions. There are four fields available on this annotation: `frequency()`, `status()`, `description()`, and `misc()`. These annotations are made available to work with [[/TCH|TCH]]. Please see the [[/TCH|TCH]] documentation for more information on their meanings.
Line 134: Line 134:
If you have questions about the JPF syntax please set the [wiki:/NetUI NetUI] documentation. In the above example there are two 'Actions'. The first is the `begin()` Action which is required on every JPF. In this case the `begin()` Action does not contain a test and puts an Report with an ABORT status on the request. Because Milton must relies on the data found in the `Report` obejct to analyzet he results of a test this is accomplised with JPF but putting the `Report` on the request and giving it a known name, `Report.KEY` and 'Forwarding' the user to a known JSP, `Report.RESULTS`. Using `Report.RESULTS` and `Report.KEY` is required for correct Milton operation because the `Report` must be forwarded to a JSP which converts the `Report` to an HTML table which can be parsed by the JUnit accessor. THis will be true of all Actions which execute test code you want analyzed. (The Milton JSPs are copied into your webapp at build time. See ControlsTestWebApps) The second action is `testInstantiation()` which creates a `Report`, instantiates a Driver, invokes the `testInstantiation(InstantiationControlBean icb)` method on the Driver, and returns the results by putting the `Report` on the request using the `Forward`. Again, the scope of this doc is not to explain JPF so if the JPF doesn't make sense please see the [wiki:/NetUI NetUI] documentation. To correctly use Milton to test your controls you will need to mimic the above Action. Also note, this test uses the Declarative Instantiation to instantiate the test control. If you want to use Programatic Instantiation please [wiki:/Controls Controls] documentation. If you have questions about the JPF syntax please set the [[/NetUI|NetUI]] documentation. In the above example there are two 'Actions'. The first is the `begin()` Action which is required on every JPF. In this case the `begin()` Action does not contain a test and puts an Report with an ABORT status on the request. Because Milton must relies on the data found in the `Report` obejct to analyzet he results of a test this is accomplised with JPF but putting the `Report` on the request and giving it a known name, `Report.KEY` and 'Forwarding' the user to a known JSP, `Report.RESULTS`. Using `Report.RESULTS` and `Report.KEY` is required for correct Milton operation because the `Report` must be forwarded to a JSP which converts the `Report` to an HTML table which can be parsed by the JUnit accessor. THis will be true of all Actions which execute test code you want analyzed. (The Milton JSPs are copied into your webapp at build time. See ControlsTestWebApps) The second action is `testInstantiation()` which creates a `Report`, instantiates a Driver, invokes the `testInstantiation(InstantiationControlBean icb)` method on the Driver, and returns the results by putting the `Report` on the request using the `Forward`. Again, the scope of this doc is not to explain JPF so if the JPF doesn't make sense please see the [[/NetUI|NetUI]] documentation. To correctly use Milton to test your controls you will need to mimic the above Action. Also note, this test uses the Declarative Instantiation to instantiate the test control. If you want to use Programatic Instantiation please [[/Controls|Controls]] documentation.
Line 165: Line 165:
If you have questions about the JWS syntax please refer to the [wiki:/WSM WSM] documentation. This should look similar to the JPF example above with a few minor changes. In this case there is a `WebMethod` called `testInstantiate()` which can directly return a `Report` to the JUnit accessor. Also note that when using Milton your `WebMethod` must take no parameters as the Milton test clients have no mechanism to pass data to the JWS. In other words, your tests must be self contained as much as possible so that they can easily be moved from one client context to another.
[[BR]]
If you have questions about the JWS syntax please refer to the [[/WSM|WSM]] documentation. This should look similar to the JPF example above with a few minor changes. In this case there is a `WebMethod` called `testInstantiate()` which can directly return a `Report` to the JUnit accessor. Also note that when using Milton your `WebMethod` must take no parameters as the Milton test clients have no mechanism to pass data to the JWS. In other words, your tests must be self contained as much as possible so that they can easily be moved from one client context to another.
<<BR>>
Line 186: Line 186:
JPF Accessors like the one above must extend `HtmlReportTestCase` to take advantage of the `assertReport` methods. The String passed to `assertReport` is the path relative to the root to the JPF which contains your test. (You can put in a fully qualified URL here if you prefer but Milton can discover the name of your host) Here we take advantage of the JPF 'actionName.do' syntax. The annotations `@Freq` and `@Status` are used for creating TCH files. You are highly encouraged to use TCH as it helps to organize the test base. For more information on TCH please see the [wiki:/TCH TCH] documentation. [[BR]] JPF Accessors like the one above must extend `HtmlReportTestCase` to take advantage of the `assertReport` methods. The String passed to `assertReport` is the path relative to the root to the JPF which contains your test. (You can put in a fully qualified URL here if you prefer but Milton can discover the name of your host) Here we take advantage of the JPF 'actionName.do' syntax. The annotations `@Freq` and `@Status` are used for creating TCH files. You are highly encouraged to use TCH as it helps to organize the test base. For more information on TCH please see the [[/TCH|TCH]] documentation. <<BR>>
Line 206: Line 206:
The above is quite similar to the JPF accessor example. Here the accessor extends `SOAPReportTestCase` which contains code to deserialize the `Report` returned by the `testInstantiation()` Web Method. The `Report` is then inspected for the test status (PASS,FAIL,ABORT), for any messages that were added to the Report, and any exceptions that were attached.[[BR]] The above is quite similar to the JPF accessor example. Here the accessor extends `SOAPReportTestCase` which contains code to deserialize the `Report` returned by the `testInstantiation()` Web Method. The `Report` is then inspected for the test status (PASS,FAIL,ABORT), for any messages that were added to the Report, and any exceptions that were attached.<<BR>>

Overview

Milton is a part test framework and utility and part methodology. It is comprised of a three parts.

  • JUnit TestCase classes (HtmlReportTestCase and SOAPReportTestCase) for invoking Milton style tests through JPF|JSP and JWS respectively.

  • A Report class which is returned by a test method and analyzed by the TestCase.

  • Support for generating client code for excercising a Control. Using a Mantis processor the Milton generator will parse Controls test files, called Drivers and generate the corresponding JPF, JWS, and JUnit accessors allowing the test author to focus on writing the test logic and get end-to-end testing virtually for free.

The Basics

The main idea behind Milton is to make it easier and faster to write end-to-end tests for the Beehive Controls framework. We want to be able to verify that all features Controls work in the two Control client contexts in Beehive: Java Page Flows (JPF) and the JSR-181 implementation of Java Web Services (JWS). "Note: Jave Web Services in Beehive are called Web Services Metadata (WSM) but use the .jws file extension." To accomplish this we have broken the tests into four pieces.

  • 1. The Control being used for testing.
  • 2. A POJO called a Driver which contains the logic which excercises the control and dertermins PASS/FAIL/ABORT
  • 3. A Client of the control (JPF, JWS, Java)
  • 4. A JUnit file which is used to access and trigger the client test methods.

A Controls test using Milton will run in the following order.

  • a. User invokes JUnit Accessor method using either TCH or standard JUnit runner.

  • b. Test method in the JUnit Accessor sends request to corresponding test method on Controls client (JPF, JWS, Java)
  • c. The Client instantiates an instance of the test Control and an instance of the Driver. The newly instantiated Control is passed to the corresponding test method on the Driver and a Report is returned. The Report is passed back to the Client and the back to the Accessor where the result in recorded.

Writing Controls Tests

There are two main ways to write tests using Milton though in the end they have the same pieces. The difference is in which of the four pieces you need ot provide.

Using Milton Client Generation

This is the recommended way for writing Controls tests. It is the the easiest and most efficient way of running end-to-end Controls tests. Using Client Generation you need only write the Control(s) needed for testing and the Driver which excercises the test control(s). Drivers are basically POJO which look similar to JUnit test cases with a few minor differences.

Here is an example Driver:

import org.apache.beehive.test.tools.milton.common.Report;
import org.apache.beehive.test.tools.milton.annotations.Milton;

import org.apache.beehive.controls.test.controls.instantiate.InstantiateControlBean;

@Milton.TestSuite
public class InstantiationDriver
{
        @Milton.Test
        public Report testInstantiation(InstatiateControlBean icb){

                Report report = new Report();

                if (icb == null) {
                        report.setStatus(Report.FAIL);
                        report.addMessage("InstantiateControlBean hcb is Null!");
                }
                else
                {
                        if (icb.echoString("hello").equals("hello"))
                                report.setStatus(Report.PASS);
                        else {
                                report.setStatus(Report.FAIL);
                                report.addMessage("icb.echoString: " + icb.echoString("hello"));
                        }
                }
                return report;
        }

}
  • 1. The Imports: In order to use Milton Client Generation you must import the Milton annotation interface, the Milton Report class, and any Control Beans that your test will use.
  • 2. The Driver Class: All Drivers you wish to have clients generated for must have the @Milton.TestSuite method. This tells the Milton processor that this Driver will need clients generated. @Milton.TestSuite has two fields: clients() which is an array of Enumerated types of support clients. Currently there are only two client types: Milton.ClientType.JWS and Milton.ClientType.JPF. By default clients is set to {Milton.ClientType.JWS, Milton.ClientType.JPF}. If you only wanted your test to run in one of those you could override it here. The second field available for override on @Milton.TestSuite is junit() which is a boolean which when set to true will cause the Milton Client Generation processor to generate JUnit accessors for your clients. If you wish to write your own JUnit accessors set this to false. As with all JWS-175 annotations you need only explicitly set the annotations fields if you wish to override the default. As in the example, simply placing @Milton.TestSuite on your class will result in the generation of JWS and JPF clients and JUnit accessors for each.

  • 3. The Test Method Declaration: All methods in a Milton Driver using Client Generation must having the following convention.
    • 1. The method must return a Report object.

    • 2. The method name must start with 'test'
    • 3. The method must have at least one paramater that is a ControlBean.

    • 4. The method must have the @Milton.Test annotation applied to it. This tells the Milton Client Generation processor that this is infact a Test method and that it must comply with the above conventions. There are four fields available on this annotation: frequency(), status(), description(), and misc(). These annotations are made available to work with TCH. Please see the TCH documentation for more information on their meanings.

  • 4. The Test Methods Parameters: As stated in above in 3.3 each test method much have at least one parameter that is of type ControlBean. The generated clients will instantiate a bean of this type and pass it to the Driver test method. There are four fields available on this annotation:

    • 1. name() which by default is blank. It is recommended that you leave the name blank. This is the name which will be used in the client code to name the ControlBean type instance and used to be passed to your test method. See documentation below.

    • 2. instantiate() is an instance of the Enumerated type, Milton.Instantiate and can be set to Milton.Instantiate.DECLARATIVE (which is the default) or Milton.Instantiate.PROGRAMATIC which will cause the client to use the java.beans.Beans API for instantiating an instance of the test Control.

    • 3. annotations() is an array of Strings which allows you to add annotations to Controls which use Declarative instantiation. annotations() will be ignored for Controls using Programatic instantiation. Currently Milton Client Generation does not support Property setting when using Programatic instantiation. If you require this for your test you will need to use method two described below.

    • 4. modifier() is used to change the java access modifier for your control. By default it is public.

    • 5. genericType() is used when you wish to use generics with your control. By default no generic type is set. Setting this to "String" will result in your control being declared as: {{{@Control

public SomeControlBean<String> testMethodName_scb;}}}

  • 4. The Test Method Body
    • 1. In the example this test expects to receive a non-null InstantiateControlBean instance from the client. The ControlBean has a method called echoString(String s) which should simply return back the String passed to it. A Report is instantiated at the beginning of the method and updated as the test progresses and is then returned.

Using Milton without Client Generation

There will be cases when the limitations in the Client Generation will impare your ability to write tests (it is generating code after all - we can't read your mind and anticipate 'all' posibilty testing scearios) or you may just prefer to write all the parts yourself. This section will describe the parts you'll need to write and provide descriptions. For those who are using Client Generation this section will be useful if you wish to understand what is generated for your tests to run.

  • 1. The Controls: You will always need to write the Test Controls no matter how you choose to write your tests.
  • 1. The Driver: When not using Client Generation it is still recommended that you use the Driver style of writing tests but leave the @Milton. annotations out. The conventions which are inforced when you use Client Generation are still encouraged when you write it yourself.

  • 2. The Client(s): You will need to write the JPF and JWS clients for your test.
  • 3. The JUnit Accessors: These are the simple JUnit files which invoke the tests on your clients.

JPF Client Example:

import org.apache.beehive.netui.pageflow.PageFlowController;
import org.apache.beehive.netui.pageflow.Forward;
import org.apache.beehive.netui.pageflow.annotations.Jpf;

import org.apache.beehive.controls.api.bean.Control;

import org.apache.beehive.controls.test.controls.instantiate.InstantiateControlBean;
import org.apache.beehive.controls.test.driver.instantiate.InstantiationeDriver;

import org.apache.beehive.test.tools.milton.common.Report;

@Jpf.Controller(
forwards = { @Jpf.Forward(name=Report.RESULTS,path = Report.RESULTSJSP) } 
)
public class Controller extends PageFlowController
{
    @Control
    public InstantiateControlBean icb;
    
    @Jpf.Action
    protected Forward begin()
    {
        return new Forward(Report.RESULTS, Report.KEY, new 
        Report(Report.ABORT,"begin() is not a test method!"));
    }
    
    @Jpf.Action
    public Forward testInstantiation()
    {
       Report report = new Report()
       InstantiationDriver id = new InstantiationDriver();
       report = id.testInstantiation(icb); 
       return new Forward(Report.RESULTS, Report.KEY, report);
    }
}

If you have questions about the JPF syntax please set the NetUI documentation. In the above example there are two 'Actions'. The first is the begin() Action which is required on every JPF. In this case the begin() Action does not contain a test and puts an Report with an ABORT status on the request. Because Milton must relies on the data found in the Report obejct to analyzet he results of a test this is accomplised with JPF but putting the Report on the request and giving it a known name, Report.KEY and 'Forwarding' the user to a known JSP, Report.RESULTS. Using Report.RESULTS and Report.KEY is required for correct Milton operation because the Report must be forwarded to a JSP which converts the Report to an HTML table which can be parsed by the JUnit accessor. THis will be true of all Actions which execute test code you want analyzed. (The Milton JSPs are copied into your webapp at build time. See ControlsTestWebApps) The second action is testInstantiation() which creates a Report, instantiates a Driver, invokes the testInstantiation(InstantiationControlBean icb) method on the Driver, and returns the results by putting the Report on the request using the Forward. Again, the scope of this doc is not to explain JPF so if the JPF doesn't make sense please see the NetUI documentation. To correctly use Milton to test your controls you will need to mimic the above Action. Also note, this test uses the Declarative Instantiation to instantiate the test control. If you want to use Programatic Instantiation please Controls documentation.

JWS Client Example:

package milton.webservices.instantiate;

import javax.jws.WebMethod;
import javax.jws.WebService;

import org.apache.beehive.controls.test.controls.instantiate.InstantiateControlBean;
import org.apache.beehive.controls.test.driver.instantiate.InstantiationDriver;

import org.apache.beehive.test.tools.milton.common.Report;

import org.apache.beehive.controls.api.bean.Control;

@WebService
public class InstantiationDriverWebService
{
    @Control
    public InstantiationControlBean icb;
      
    @WebMethod
    public Report testInstantiation() throws Exception
    {
        InstantiationDriver id = new InstantiationDriver();
        return id.testInstantiation(icb);
    }
}

If you have questions about the JWS syntax please refer to the WSM documentation. This should look similar to the JPF example above with a few minor changes. In this case there is a WebMethod called testInstantiate() which can directly return a Report to the JUnit accessor. Also note that when using Milton your WebMethod must take no parameters as the Milton test clients have no mechanism to pass data to the JWS. In other words, your tests must be self contained as much as possible so that they can easily be moved from one client context to another.

JUnit Accessor Examples:

import org.apache.beehive.test.tools.milton.junit.HtmlReportTestCase;

import org.apache.beehive.test.tools.mantis.annotations.tch.*;

public class TestInstantiation extends HtmlReportTestCase
{
    public TestInstantiation(String s){super(s);}
    
    @Freq("checkin")
    @Status("active")
    public void testInstantiation() throws Exception
    {
        assertReport("/controlsWeb/milton/pageflows/instantiate/InstantiationDriver/testInstantiation.do");
    }
}

JPF Accessors like the one above must extend HtmlReportTestCase to take advantage of the assertReport methods. The String passed to assertReport is the path relative to the root to the JPF which contains your test. (You can put in a fully qualified URL here if you prefer but Milton can discover the name of your host) Here we take advantage of the JPF 'actionName.do' syntax. The annotations @Freq and @Status are used for creating TCH files. You are highly encouraged to use TCH as it helps to organize the test base. For more information on TCH please see the TCH documentation.

import org.apache.beehive.test.tools.milton.junit.SOAPReportTestCase;

import org.apache.beehive.test.tools.mantis.annotations.tch.*;

public class TestInstantiation extends SOAPReportTestCase
{
    public TestInstantiation(String s){super(s);}
    
    @Freq("checkin")
    @Status("active")
    public void testInstantiation() throws Exception
    {
        assertReport("/controlsWeb/milton/webservices/instantiate/InstantiationDriverWebService.jws", "testInstantiation");
    }
}

The above is quite similar to the JPF accessor example. Here the accessor extends SOAPReportTestCase which contains code to deserialize the Report returned by the testInstantiation() Web Method. The Report is then inspected for the test status (PASS,FAIL,ABORT), for any messages that were added to the Report, and any exceptions that were attached.

In both examples, the test method (WebMethod and Action) names are all the same. This helps trace which methods are associated with each other. This convention is not required but is definitly encouraged. When using Client Generation you will get tests generated which look very similar to the above examples (JPF, JPF, JUnit Accessors).

Controls/TestingControls/Milton (last edited 2009-09-20 23:24:28 by localhost)