Code Generation in MyFaces

The Issues

In JSF, there are a number of classes and configuration files that need to be kept synchronized; a change to one requires updates to other artifacts. In addition, there are a number of places where extremely repetitive, "boilerplate" code or configuration is needed.

Keeping the files in sync manually is a lot of work, and it is easy to make mistakes. There have therefore been a number of attempts to simplify things; this page documents the various approaches and some of the proposals for the future.

The Artifacts

The artifacts in question are:

Each UIComponent class typically has a number of "jsf" properties, requiring getter and getter methods. The implementation of these methods is almost identical for each property. Each UIComponent class also needs saveState and restoreState methods that are very simple, and vary only due to the set of properties available on the component class. Preferably, comments need to be present on the class and the properties that are similar or identical to the documentation available in the jsp taglib files.

?? converters ??

?? Validators ??

A jsp taghandler class must exist for each UIComponent. In most cases these classes are extremely repetitive to code and simply create a component and copy values from the jsp tag onto the component. The taghandler must have a property (setter/getter) for each "jsf" property on the corresponding component class.

One or more jsp taglib files needs to be created, with an entry for each jsp taghandler class, and nested attribute entries for each "jsf property" of each taghandler class. Ideally, documentation needs to be present on each tag entry and each attribute entry. Due to limitations in the taglib format, even when two uicomponent classes are subclasses of the same base, the configuration for the attributes they inherit from the shared parent must be duplicated.

The faces-config.xml file needs to declare each component, each converter and each validator, and map components to renderers. Additionally, many tools derive their information about components from metadata in the faces-config.xml file. The JSF-Metadata EG is specifying the format for this, even though this EG is rather dormant currently, in the end this might help significantly with tool-adoption.

For facelets support, a facelets configuration file is needed which is similar to the faces-config.xml file.

For components with method-bindings other than valueChangeListener, action, actionListener, a facelets tag handler class is needed.

Previous Solutions

Current myfaces core (1.1.5)

MyFaces Core 1.1.5 and a few preceding releases simply keep things in sync manually. Adding an attribute to a component means writing the setters and getters on the uicomponent, doing the same on the taghandler, then updating the taglib files. Adding a component means creating uicomponent and taglib classes, ensuring all the datatypes match, updating faces-config.xml etc. However myfaces core is an implementation of the jsf spec, so now that the spec has been completely implemented, components do not get added or their apis modified so this is less of an issue. Documentation improvements are still somewhat clumsy as they may need to be done both in the jsp taglibs and the relevant .java file.

Facelets config files are maintained by hand too, outside of the myfaces core project.

To avoid the horrible duplication of entries in .tld files, there is a simple xsl-stylesheet-based solution to allow shared attribute definitions to be referenced from the tld entries for each component.

Current Tomahawk (1.1.6)

This works just like myfaces core 1.1.5, ie everything done by hand. But, unlike core, new components, validators and converters do regularly get added to tomahawk and component apis do change. The effort to keep all this in sync is therefore significant.

Again, facelets support is currently managed outside the project; in this case, via pages maintained on the wiki.

Older Myfaces Core 1.x

At one time, a code generator was built for myfaces core 1.x. The approach was for xml configuration files to contain meta-data about what jsf properties each component had. Each component class and taglib class (checked in to svn) had special markers like === BEGIN GENERATED CODE === and === END GENERATED CODE ===. When any jsf property changed, an ant task could be run which would erase all code within the marked blocks, and then regenerate it using the (new) metadata; the java source code was modified "in place", and the result was then checked back in to svn. Presumably taglib and faces-config files could also be generated(?).

The result was that code checked out of svn was always complete, compilable, normal code - except for these special comments, within which code was not editable (or at least would be wiped out next time code was regenerated). Bugfixes would go directly into these classes, but api changes must be made in the metadata files instead, and the code-genration

This code eventually fell into disuse, and people started making manual changes to the "generated" sections. This code generation approach was eventually abandoned, more through disinterest and lack of knowledge than any deliberate decision.

One of the last developers to work on the old code-generation framework commented that it was "very painful". Not sure whether this comment was about the basic concept of this approach, or just the implementation.

Trinidad

The trinidad project, based on code written at Oracle, created a build process in which extensive metadata is stored in xml config files. Template files, resembling java but with special markers, are checked in to a "templates" directory. A processing step then merges the metadata into the templates to generate complete classes. The resulting compilable java classes are *not* checked in to svn.

The "processing step" was initially an Ant task, later became a maven1 plugin, and then a maven2 plugin (called trinidad-faces-plugin).

This process has been used for all trinidad releases (both 1.1.x and 1.2.x) to the current date.

The build plugin does have the ability to skip the code-generation for some classes, allowing specific ones to be hand-written.

Myfaces core 1.2.0 and 1.2.2

When the myfaces core 1.2.x series was branched and developed, the existing trinidad component build system was adopted. The old myfaces core component classes became templates, and metadata files were created to define their properties. The corresponding hand-written code was removed from the template files, as it was then generated instead.

Tobago

Tobago was mentioned only very briefly during the discussions. No-one seemed to know how Tobago manages this issue.

Further Discussions

During discussions about the future release plans for Tomahawk, significant debate occurred about the best way to address the issues listed at the top of this page. The idea of adopting the current trinidad/myfaces-1.2.x system did not meet with universal approval, and this page attempts to list the main points made during those discussions.

The discussion thread commenced on myfaces-dev on 30 Jan 2008.

Using templates to generate the artifacts (aka The Trinidad approach)

Some discussion occurred about how Trinidad dealt with these issues. There was quite a bit of misunderstanding, however.

Hopefully a trinidad developer can fill in this section properly.

Trinidad-faces-plugin input files are modelled on the idea of a "mini faces config file". It appears that trinidad has quite a deep "class hierarchy" of templates, ie templates that generate abstract classes, and other templates that then generate subclasses of those generated abstract classes.

Some component classes do have significant logic internally, ie they are not just dumb data-structures. This logic is in the templates, ie does not become "real" java code until after the code-generation step.

It was pointed out that templates can be used to define behaviour that applies to sets of components even when they do not share a common ancestor. One example is the Tomahawk "forceId" feature. Components that should share a feature can just share a template, causing them to get copies of the same code.

This isn't infinitely flexible, however; a component can have only one template so it gets all of the code from that template or none (eg if the template implements forceId and enabledOnUserInRole, it cannot get one without the other. It is possible to use a hierarchy of templates to define a hierarchy of "real" ancestor classes which can give some kind of control over this, but as java is a single-inheritance language this approach cannot really provide "mixin" functionality.

One reply commented that this looks a lot like aspect-oriented-programming, and that if it is needed then maybe a real AOP tool should be used.

Generating base classes instead of templates

It was suggested that code-generation for component classes could just generate an abstract parent class, and that a "real" component class would then subclass it. The parent would never be modified by developers, and the child class would never be modified by code-generation.

It was not decided whether the parent-class would be checked in or not - there were several statements against checking in generated classes.

Having a base class does allow code generation to insert package-scope variables and methods into the generated class, which the concrete child would have access to due to being in the same package. These vars/methods would not be visible to subclasses, however, so would not "pollute" the api.

One drawback is that this generated class is visible in the public ancestry of the concrete class, distorting the "clean" hierarchy. An opinion was expressed that "-1; .. the resultant OO structure is not clean".

This approach is feasible for libraries like Tomahawk and Trinidad and the MyFaces Impl.

Note that (in a feature that may surprise some Java developers) it appears quite valid for a class to have a package-scoped parent; the class can still be subclassed or instantiated from outside the package. It inherits public and protected members from its package-scoped parent which can be called as normal. The only constraint is that it cannot be cast to its parent type, as this is not accessible (although the Class object for that type can still be obtained). It is not yet known whether inserting such a package-scoped ancestor class into the ancestry of a component class in the javax.faces package scope would be acceptable to the JSF TCK or not. If the TCK accepts this, then the approach of generating a base class could also be applied to myfaces core components.

In the practice the problem is that jsf core (myfaces and ri) uses reflection to set the attributes and the following case fail:

//base component class from api
public class BaseComponent extends Object
{
    //......//
}

//package scope class
abstract class AbstractComponent extends BaseComponent
{
    private String value;

    public String getValue()
    {
        return value;
    }

    public void setValue(String value)
    {
        this.value = value;
    }
}

public class RealComponent extends AbstractComponent
{
    //......//
}

If an instance of RealComponent is created, you can access to the public api defined on AbstractComponent. But if you use reflection to call getValue or setValue using a code like this:

    public Object getValue(BaseComponent component){
        Object resp = null;
        try
        {
            BeanInfo beanInfo = null;
            try
            {
                beanInfo = Introspector.getBeanInfo(component.getClass());
            }
            catch (IntrospectionException e)
            {
                e.printStackTrace();
            }
            PropertyDescriptor[] propertyDescriptors = beanInfo
                    .getPropertyDescriptors();

            HashMap<String, PropertyDescriptor> _propertyDescriptorMap = new HashMap<String, PropertyDescriptor>();
            for (int i = 0; i < propertyDescriptors.length; i++)
            {
                PropertyDescriptor propertyDescriptor = propertyDescriptors[i];
                if (propertyDescriptor.getReadMethod() != null)
                {
                    _propertyDescriptorMap.put(propertyDescriptor.getName(),
                            propertyDescriptor);
                }
            }

            PropertyDescriptor pd = _propertyDescriptorMap.get("value");

            Method readMethod = pd.getReadMethod();

            readMethod.setAccessible(true);
            Object[] EMPTY_ARGS = new Object[0];

            resp = readMethod.invoke(component, EMPTY_ARGS);

        }
        catch (Exception e)
        {
            e.printStackTrace();
        }        
        return resp;
    }

Fail with the following exception

java.lang.IllegalAccessException: Class javax.faces.component._Util can not access a member of class org.apache.myfaces.test.AbstractComponent with modifiers "public"
        at sun.reflect.Reflection.ensureMemberAccess(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at javax.faces.component._Util.getValue(_Util.java:54)
        at javax.faces.component.BaseComponent.getValueReflection(BaseComponent.java:32)
        at javax.faces.other.ComponentTest.main(ComponentTest.java:14)

This behavior is a JDK bug:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4071957 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4533479

One possible workaround is put the following line before invoke:

Or make AbstractComponent public.

The conclusion is that the abstract base class should be public if and only if it has in his body a attribute definition available on the tld. If the abstract base class has some different code it can be package scope. This behavior discard this approach for myfaces core api!

Meta-data would of course still be maintained in configuration files of some sort, external to the generated code.

With this setup, we have the question of which source directory the generated base class gets written to. Option (1) is to write it to a generated-source dir, option (2) is to write it to the normal source dir.

Question: When there are multiple source trees for a module, does maven build against them all simultaneously? That is, if a normal source file references a source file in the generated-source tree which itself references a source file in the normal source tree, does this work?

Response: One successful example is this hierarchy used for t:schedule

javax.faces.UIComponentBase


myfaces core api org.apache.myfaces.custom.schedule.AbstractUIScheduleBase


on src/main/java org.apache.myfaces.custom.schedule.UIScheduleBase


generated on target/maven-faces-plugin/main/java org.apache.myfaces.custom.schedule.UISchedule


on src/main/java org.apache.myfaces.custom.schedule.HtmlSchedule


generated on target/maven-faces-plugin/main/java

Even if compiling across multiple source trees does work(it does), it could be rather confusing.

Other possible solutions are this:

  1. write the generated base classes into the normal source tree, or
  2. put the hand-written child class into the template tree, and have it be copied unaltered to the generated-source tree.

COMMENT

Generate in src/main/java or in target/maven-faces-plugin/main/java is transparent for the IDE. If we generate in src/main/java, the generated code will mix with hand written code, if we translate generated code to a separate directory on src/main/java (for example src/main/java-generated) technically it is equivalent to generate in target/maven-faces-plugin/main/java (the difference is if we do mvn clean, the code in target is deleted).

Putting the hand-written child into the template dir brings back the original issue with templates though: the code-generation step would need to be run after every modification to that hand-written code. And IDE support would not be available for editing that code.

Svn-ignore flags could be set so that these files are not reported by svn, if they are not checked in. There still is some risk, however, of these generated classes accidentally getting checked in.

With this approach, there is also the issue that a concrete class needs to be hand-created and checked in, even if there are actually no customisations needed. However, writing an empty class should not be too much of a problem. Possibly a tool could detect this case, though, and create the concrete class definition too.

This comment was made about this approach:

For what it's worth, what you're describing here is the Generation Gap
pattern.   I've got a lot of experience using it with Cayenne over the
years (and WebObjects years before that), and it's effective.   It's
sometimes painful to write and maintain your base class in a
templating language, though, but once your base class is done, you
don't need to mess with it again, so this is a one-time painful
experience.   And yes, you would typically generate an empty concrete
subclass whenever one doesn't exist (the first time code generation
runs for a given component).

I check the code thus generated into the repository.   I also keep the
generated code in a subpackage called *.generated off the concrete
subclasses's package.

An annotation-driven approach

It was suggested that the component, converter and validator classes should be written as normal java classes, ie in the manual style currently used for myfaces 1.1.x and tomahawk. However some kind of annotations (whether java1.5 annotations, xdoclet tags, or other) would be used to mark parts of the class. These classes would be checked in to svn as normal.

A processing tool would then scan the source, and use the annotations to drive the generation of jsp taghandlers, tld files, faces-config files, facelets config files, etc. No external meta-data files would be kept; all information necessary to generate taghandlers and the jsf configuration files would be in the source code files instead.

This does require more manual coding than the trinidad-faces-plugin approach. However it does also mean that all code checked in to svn will compile immediately, without requiring any processing steps (no code in the core jsf classes depend upon the existence of taglib classes or config files).

Debugging and bugfixing should be simplified with this approach, as there are no special generated-source directories that IDEs need to be told about, and the source to be fixed is "normal", not template code that is not actually a real class. Those classes that would be generated (jsp taghandlers) are very seldom of interest for debugging or development.

Open questions were whether

  1. writing the property getter/setters is too much work, and b. how saveState/restoreState can be sensibly done without code generation

Some people expressed the opinion that (a) was not a significant issue, others see this as an issue if getters look like they look today (this could easily be changed, however, by introducing a few utility methods which would make one-line getters out of the rather complicated approach today).

Item (b) is indeed an issue, and would need to be addressed somehow. The implementation of save/restore methods is very simple, but does require listing all the jsf properties of the class correctly, and in the same order in each method; manual implementation of these seemed to be generally viewed as undesirable. One possibility is to move towards the Trinidad state-saving approach in which state is "externalised" and the save/restore methods are much simpler. This would however not work with components deriving from JSF-API base-classes (Trinidad only implements the base-interface), as then the restoreState/saveState approach is already predefined. Changing the approach in the base-classes might be possible, but it will be hard to do this, due to API restrictions (however, it might be worth investing the time to check this, it might help with performance).

Speculation also occurred over whether reflection, or code-generated "helper" classes could be used to implement save/restore methods. Nothing conclusive came of this though, except that the obvious solution had catches. Using private variable access via reflection is probably not reliable due to classloader security constraints, etc.

A question was raised about why state isn't retrieved from the attributes map - this cannot be used, however, cause it would use reflection internally and call the getter of the method, if a value is not directly stored in the attributes map. As soon as the getter is called, after the check for the local value returns null, you will get back the value from a value-expression, and you do not want to save this value in the state, as the value-expression itself is already stored!

Annotation approaches

Note that the word "annotation" above is being meant in the general sense of storing metadata in the source file (as the first paragraph in the section points out). Using Java's built-in Annotation language feature is just one option, as discussed below.

The options are:

General comments on real Java Annotations

Some criticism of this was that expressing all the necessary metadata will be pretty tedious, especially as some of the artifacts (e.g. faces-config description, tld-description and faces-config meta-data for tool-support) can contain or are written in XML syntax; this would need to be converted to an annotation based syntax. Additionally, there are peculiarities with annotations which do not make it easy to use them as a full replacement for a complex XML configuration. One, they do not allow for a null-value as a default-value, two, if you want to build a complex structure with them, the syntax is not very intuitive.

I would argue that the meta-data information itself is abstract; it is then made concrete as xml files in some cases, but as java code in others. So it isn't right to think about "how do we write a faces-config file in annotations", but instead simply how to represent the complete set of necessary information. Yes annotations are somewhat less flexible than xml, but the data to be represented isn't that complex. Some experimenting would be necessary to determine whether a reasonable syntax could be found or not.

Using real Java Annotations, retention=Class

In this approach, the java compiler would include information in the generated .class file about the annotations in the source. A tool could then post-process these class files to find this information.

IDEs have code-completion support for annotations, so working with them could be quite pleasant for users of these types. There is no runtime penalty, because the annotation information is not loaded into Class objects at runtime.

However the code would have to be compiled with java 1.5 or later, and the bytecode output would only be compatible with java 1.5 or later JVMs. Possibly something like RetroWeaver could be used to make the code 1.4-compatible (throwing away the annotation info after it has been used is fine) but this is a little clumsy.

It would not be possible for checked-in code to depend on generated code, as the checked-in code must be compiled first before its .class files can be post-processed. We would also be generating source after the first compile pass has finished, meaning compilation needs to be done in two passes. Not sure how easy this would be to set up with maven.

Javadoc attached to annotated entities would not be available, as that is not present in the generated .class files. That means that taglib entries etc. cannot include documentation taken from javadoc in the source files. It is possible to define documentation using String-typed annotation attributes, but that would be pretty clumsy in practice.

Using real Java Annotations, retention=Source

In this approach, the java compiler would simply ignore the annotations it finds in the source. However other tools can be used to inspect the source code for these annotations.

IDEs have code-completion support for annotations, so working with them could be quite pleasant for users of these types. There is no runtime penalty, because the annotation information is not even present in the .class files.

However the code would have to be compiled with java 1.5 or later. And unfortunately, the "-target 1.4" option cannot be used with "-source 1.5", so the bytecode output would only be compatible with java 1.5 or later JVMs.

The code then needs to be processed with some kind of source-code analyser. Sun's JDK comes with a tool called "APT" for this purpose, but it looks pretty ugly and non-portable. Possibly Eclipse provides something; plugins do a lot of source-code scanning. As noted in another section here, there are other source-code parsers available too.

There is no problem with checked-in code depending on generated code, as the code-generation phase can run before the compilation phase (scanning runs against the source, not the binary output).

Javadoc attached to annotated entities would probably be available, depending on how the selected source-code scanner works. So taglib entries etc. can include documentation taken from javadoc in the source files; this would seem quite convenient for development.

Using real Java Annotations, retention=Class

In this approach, specially-formatted comments, eg /** @jsf:component */ are used in the code.

This has a long history; xdoclet has worked this way for ages, and the Plexus framework that Maven is built on also does this.

There are a couple of well-known java source code parsing libraries. They are open-source, and pure java so entirely portable. It should be easy to get access to the rest of the javadoc block, so mapping javadoc comments on classes or properties into taglib doc is simple.

However it is a new format for users to learn. And there would be no IDE auto-complete help for comment formats

Going back to the old Myfaces Core generation approach

Some interest was expressed in going back to the original code-generation approach, where classes were checked in to the normal source dirs, with special markers. A build process could be run which used metadata files to modify the marked bits of source files in place, and the results checked back in again. This process would only need to be run when config is changed.

Others expressed concern that metadata and source would get out of sync if the code-regeneration process was not part of every build cycle.

A number of people thought that this approach was just inelegant.

It was pointed out that if generated classes are checked in, and then the original configuration referencing that file is removed, the checked in file will remain unless manually removed.

A somewhat more sophisticated variant on the original design was suggested, where the code markers look something like java1.5 annotations rather than just simple BEGIN and END strings, and that a java source-code parsing library be used to properly analyse the input and do the insertion of generated code into the java file in a smarter way. This proposal came with some sample code that is too long to embed here directly. It was questionable if this approach would change anything for the better - the developer could still ignore the annotations.

Links to java source parsers:

Combining annotations with a code generator approach

   1 @Component(
   2   type = "...",
   3   family = "...",
   4   rendererType = "...",
   5   tagClass = "...",
   6   events = {
   7     @ComponentEvent(
   8       type = "...",
   9       phases = { "...", "..." }),
  10     ...
  11   )
  12 public abstract class MyComponent extends UIXComponent
  13 {
  14   @ComponentProperty(
  15     description = "or is this better in javadoc? -- harder to code the generator, but easier to document",
  16     extensions = {
  17       @PropertyExtension(name = "preferred", value = Boolean.TRUE),
  18       @PropertyExtension(name = "...", value = "...")
  19     })
  20   private String _foo;
  21 
  22   @ComponentPropertySkel
  23   public abstract String getFoo();
  24 
  25   public void broadcast(FacesEvent event)
  26   {
  27     if ("bar".equals(getFoo()) {
  28     // custom code
  29     }
  30   }
  31 }

This would be in a pre-processing folder of maven (not src/main/java). The plugin can then take this file an build the concrete class from it.

This is basically the same as the trinidad approach but it merges the xml with the template.

Other notes

Some interest was expressed in passing about making jsf component setters/getters use java5 generics.

It was pointed out that the trinidad-faces-plugin already has the ability to write its generated classes to anywhere that is desired, so writing to "source" or "target" directory is no problem technically.

One of the high priorities is not creating a barrier to entry for new developers, ie keeping things simple for people who first start dipping into the code.

Questions were asked about the maven2 eclipse:eclipse goal, and whether it automatically adds generated-source directories as source folders to an eclipse project. The answer is yes, but some people appear to have had problems with this.

Some frustration was expressed with the existing "shared" code system, which also resembles code-generation, and that this makes debugging and related tasks awkward. The concern was that code-generation as done by trinidad-maven-plugin would create similar (but even more extensive) problems.

Base for decision

in the following, we should establish a base for a decision - the following options have been outlined:

  1. Using xml-config files and component templates to generate the artifacts (aka The Trinidad approach)
  2. Generating base classes instead of templates from xml-config-files
  3. Use a java-class with annotations to generate the artifacts
  4. Going back to the old Myfaces Core generation approach of generating from xml-files right into the actual component-class with markers (either doc or annotations)

priorities:

in the following, we list our priorities: