Made some progress on the Documentum-Cocoon stuff over the Christmas break. I’m working on a piece that will allow me to dynamically build charts and graphs based on data stored in the docbase. Here’s what I’ve done and what’s left to do:

Step 1. Figure out how to build a bar chart in SVG

First, I needed to figure out how to build a chart using SVG. I found some Java code that builds charts using SVG but what I was looking to do was build the SVG programmatically using XSLT to transform my source data which will be XML coming from my Documentum Transformer.

I needed something I could use to graphically draw the chart to get me started on the SVG. I downloaded a drawing tool called Sodipodi for this. It worked great. It stores everything as SVG. So all I had to do was draw my chart and save it to get a nice XML file to start with.

Step 2: Create the SVG with XSLT

Now that I had a static SVG file I needed to create XSLT that would take source data XML and transform it into the bar chart SVG. First, I copied my static SVG file into my Cocoon install to make sure it was happy rendering it into JPEG and PNG. It was. Next, I downloaded the Batik binaries. I wanted a tool I could use to quickly view my SVG file as I tweaked it. Batik comes with an SVG viewer called Squiggle. In WSAD, I created a test XML source data file, copied my static SVG file from Step 1 into a new XSLT file, tweaked, and then viewed the results in Squiggle.

The first thing I did was add comments so I could figure out which part of the SVG did what. In Sodipodi I had set every object’s “id” attribute, so correlating the XML to the object it was describing was pretty easy. Adding comments made it easy to rearrange blocks of XML to work how I needed it to in the stylesheet.

The next thing I did was clean up the SVG by rounding off all of the numbers Sodipodi had used for the coordinates of my shapes. Obviously with a design tool everything has got to be very precise, but for what I needed, whole numbers worked just fine and it made the XML easier to look at.

I then figured out the algorithms I’d need to use to figure out how to dynamically size and position the bars and bar legends in the chart.

Finally, I replaced the hardcoded values with XSLT variables. I then updated my source XML data with new values, transformed, and voila: Dynamically generated bar chart.

Step 3: Grab the data from Documentum on a schedule

The data I need to chart is in Documentum. It changes over time. Because it’s historical data I need to get a snapshot from Documentum on a schedule and store the result somewhere. Then, my chart will be built on-demand via a Cocoon pipeline using the data that’s been captured for a given time period.

I already have a pipeline that gets the snapshot from Documentum. All I need to do is execute that pipeline on a schedule and store the result somewhere. I’ve tested out the “cron” feature of Cocoon and it works great. It allows you to execute a pipeline on a schedule. A perfect fit.

Now what I need is to put my snapshot data somewhere. I’m planning on using Xindice. It’s an XML database embedded in Cocoon (you can use a separate Xindice install if you want). My pipeline will pull data from Documentum, transform it (I really only need to store a subset/summarized cut of the data), and store it in Xindice.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to stick data in Xindice from a pipeline.

Step 4: Build the chart based on the historical data

Now that I’ve got my historical data summarized and sitting in Xindice, I need to create a pipeline that will query Xindice for the data and transform the query result using the SVG I build in Step 2.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to query XML data from Xindice using XPath. The rest should be a piece of cake.

The result

Once this is in place, I’ll have an approach and some reusable code I can use to capture and chart source data from Documentum.

Here‘s a new minor release of the Documentum-Cocoon integration components. It includes the reader and generator components I referenced a while back and a minor tweak to how I’m handling namespaces. (I’m not convinced I’m handling them the way I should, BTW). I’ve mentioned needing to develop a Documentum-specific protocol but now that I have a reader and a generator, that doesn’t seem as important–readers and generators could be used as “internal” resources within a pipeline to retrieve things like XSL from the docbase to perform transformations.

I’m a little torn on what to work on next regarding this integration. I’d like a more elegant way to handle sessions. I can add single sign-on fairly easily so that when called from a WDK application the credentials would be passed to Cocoon and then used to create a session as the currently logged in Documentum user.

I’d also like to migrate the administrator dashboard prototype I’ve developed to use the Cocoon Portal UI framework.

Another idea is to use Cocoon scheduler to wake up periodically, execute a pipeline that reads data from Documentum, and persist that data either back to Documentum or into the Xindice XML database. This could be used to run historical reports, for example.

At Momentum I ran across a presentation by a fellow doing XML pipelining with Cocoon and Documentum. We exchanged information. If he posts his components for public consumption I’ll link to them.

Got a Reader and a Generator added to my Cocoon-Documentum integration package last week. I also tested out the sendmail transformer that comes with Cocoon. Now, I can grab any file from Documentum from Cocoon using the Reader and then stream it to the browser or email it. If I want to transform an object stored in Documentum, I can use the Generator to snag it and then use it as I normally would in any other pipeline. For both I’m using the same protocol Documentum uses to reference documents through their “virtual link” support which is /<docbase>:<full path to content>.

I haven’t made the Reader or the Generator available yet. I’ll add it to the Transformer package when I get a chance.

What’s left to do is create a protocol. That will allow me to use any Documentum object anywhere in a pipeline rather than just being restricted to Readers and Generators. That means I could store my XSL stylesheets as objects in the Documentum repository instead of on the file system if I wanted/needed to, although that would degrade pipeline performance. 

Finally got the Documentum Cocoon Transformer cleaned up in WSAD to make it easier for me to share with others. Here is version 0.1. I’ve released it under the Apache License.

It isn’t anything fancy. Very little code. There are probably other Cocoon-Documentum integration points that would be useful but I haven’t knocked any out yet.