This page documents aspects around images in Apache FOP. Older design documentation can be found here: http://xmlgraphics.apache.org/fop/design/images.html

Current status

Some of the content below is slightly out-dated. The "current problems" are mostly "past problems" now, after the image loader framework in XML Graphics Commons has been introduced. Performance and memory consumption has been improved as expected. Still, the image handling in the various renderers is still done in different ways. As an example: Barcode4J currently makes calls against the Graphics2DAdapter interface, the ImageAdapter interface, the PSRenderer class and can still use the fallback via SVG. The coupling is too high. The PDFRenderer also still has a slightly different approach at image handling than, say, the PSRenderer. Now, with the new intermediate format, all the code that is directly dependent on the Renderer interface becomes a problem for code reuse.

Unification of image handling is being worked on as part of the implementation of the new intermediate format (AreaTreeIntermediateXml/NewDesign). The proposal is described on the ImageSupport/ImageHandler page.

Current problems

Certain renderers can embed images directly (JPEG, EPS and certain TIFF subformats, for example) but other renderers still require a decoded bitmap image. Currently, the cache only provides the first requested variant of an image. If the PDF renderer rendered an FO file with a JPEG image and then the same document is rendered with the Java2DRenderer, there will be a problem because JPEGImage loaded the original data and did not decode the JPEG image.

Another problem is with color spaces. Details here.

Format support matrix

The following matrix tries to show all the possible combinations. A more graphical view of the whole thing can be seen here (SVG, 45KB).

JPEG

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

1:1 embedding

PostScript

1:1 embedding

Requires PostScript Level 2

Java2D

decoded bitmap

PCL

decoded bitmap

AFP

decoded bitmap

SVG

referenced or RFC2396 data URL

RTF

1:1 embedding

1:1 embedding through FOP's own code. No support for decoding JPEG images through an image library, yet.

PNG

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

decoded bitmap, possibly 1:1 embedding

PostScript

decoded bitmap, possibly 1:1 embedding for PS Level 3

Java2D

decoded bitmap

PCL

decoded bitmap

AFP

decoded bitmap

SVG

referenced or RFC2396 data URL

RTF

1:1 embedding

BMP

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

decoded bitmap

PostScript

decoded bitmap

Java2D

decoded bitmap

PCL

decoded bitmap

AFP

decoded bitmap

SVG

referenced or RFC2396 data URL

RTF

1:1 embedding

GIF

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

decoded bitmap

PostScript

decoded bitmap

Java2D

decoded bitmap

PCL

decoded bitmap

AFP

decoded bitmap

SVG

referenced or RFC2396 data URL

RTF

decoded bitmap

TIFF

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

decoded bitmap

1:1 embedding for CCITT encoded images

PostScript

decoded bitmap

1:1 embedding for CCITT encoded images (NYI)

Java2D

decoded bitmap

PCL

decoded bitmap

AFP

1:1 embedding for CCITT encoded images

SVG

referenced or RFC2396 data URL

RTF

decoded bitmap

1:1 embedding through the help of Batik's TIFF codec plus FOP's own code.

SVG

fo:instream-foreign-object and fo:external-graphic

Renderer

required/preferred variant

Comments

PDF

native conversion with Batik

PostScript

native conversion with Batik

Java2D

native conversion with Batik

PCL

conversion to bitmap with Batik

HP/GL Graphics2D implementation only for the simplest of SVGs available

AFP

conversion to bitmap with Batik

GOCA implementation in the works

SVG

referenced or embedded

RTF

conversion to bitmap with Batik

Output formats (like PCL and RTF) for which no native conversion is available we need an alternative to provide the SVG as a bitmap image. This is currently implemented in AbstractGenericSVGHandler and, for RTF, in SVGConverter.

For PDF, it would be interesting to have a native picture painted into a Form XObject so such an image can be preprocessed and more easily reused. The difficulty there are features like links which would need to be handled separately since they are not part of a Form object.

Similary for PostScript, the SVG could be rendered as an EPS file which could be reused within the document.

EPS

fo:external-graphic only

Renderer

required/preferred variant

Comments

PDF

embedded

PDF support is deprecated and not supported by newer Acrobat Readers

PostScript

embedded

Java2D

not supported

PCL

not supported

SVG

not supported

RTF

not supported

If we ever have a PostScript interpreter available to FOP we can support EPS images for other output formats. An alternative could be to extract the TIFF previews provided by certain EPS images but this is better solved through a better suited image format.

The FOray project has the beginnings of a PostScript interpreter with a proof-of-concept implementation for rendering graphics. But making it usable would take a lot of work.

MathML

fo:instream-foreign-object and fo:external-graphic

MathML is internally converted to SVG in the MathML extension and subsequently handled as such. So see the SVG section for details. Same problems, too. The alternative is to render MathML directly using Java2D.

One small issue here: a math expression usually has a baseline. This baseline should be aligned with the FO baseline.

Barcode4J

fo:instream-foreign-object only

Renderer

required/preferred variant

Comments

PDF

painted using Java2D or internally converted to SVG

PostScript

internally converted to EPS

Java2D

direct painting to Graphics2D

PCL

direct painting to Graphics2D

AFP

direct painting to Graphics2D

See Bugzilla #41995 for an alternative using BCOCA

SVG

internally converted to SVG (NYI)

RTF

internally converted to bitmap

The new FOP extension is available since Barcode4J 2.0alpha1.

Other foreign XML formats

The easiest way is to convert to SVG internally and let the renderers handle that format. Examples for this section: Example plan extension, JCharts support etc.

Better is to have those extension work directly on Java2D which enables to bypass Batik and speeds things up.

Requirements for the whole solution

Random thoughts

It might be good to separate the image dimension object for the layout process from the actual decoded image, thus providing separate caches for both. [DONE]

For high-volume PostScript environments (or PPML) it might be worthwhile not to fully load images at all but to simply insert resource placeholders (DSC comment %%IncludeResource) into the stream. This would speed up the rendering process considerably for environments where such an approach is possible. [DONE]

If it is known which renderer the document will be rendered to during the layout stage, the images could be loaded in a separate thread after the dimensions have been determined while the actual layout continues. [ignored for now]

We need to get rid of our byte array approach for storing decoded images. This should be done entirely using Java2D/AWT means, i.e. RenderedImage/BufferedImage. [DONE]

ImageSupport (last edited 2009-09-20 23:52:19 by localhost)