It’s hard for me to lend much credence to Forrester’s Vendor Scorecard for Alfresco, recently released as part of its Forrester Wave report on ECM Suites. The reason is that I don’t believe too many people are looking to buy an “ECM Suite” at least as Forrester defines it.

Forrester doesn’t seem to care that most of us have moved past the legacy definition of ECM as an all-encompassing, single-vendor software suite. But as the “Suite” term implies, Forrester’s report is slanted toward gigantic vendors who offer every piece of technology even remotely related to ECM.

This is in direct contrast to what Optaros clients ask us for. They want agility, speed of implementation, product innovation, and freedom of choice. Many are implementing Web 2.0 and Enterprise 2.0 solutions which work better with the concept of “ECM as a service” rather than “ECM as giant, complex software offered by a single vendor who has cobbled together their offering from acquired and poorly integrated products”.

Given that, it is odd that Alfresco would be measured with the same scorecard used to evaluate an “ECM suite”, but that’s what they did. They judged Alfresco unfairly in a couple of places and generously in others. Really, though, the report made me realize Forrester needs to come up with a better way to segment and evaluate the market because I’m not sure people are in the market for the product that would score well against their scorecard.

Let’s take a look at the report and you’ll see what I mean. Specifically, I’m looking at the Vendor Scorecard spreadsheet for Alfresco. I’ll start by calling out what they got right.

Alfresco scored a perfect 5 for “Platform readiness”. Unfortunately for Alfresco, it’s only worth 2.5% of the total score. This set of criteria favors suites that accomplish more with fewer disparate sub-systems or products so it makes sense that Alfresco would do well here.

Alfresco took a ding on E-forms and Email archiving scoring 1.0’s for both criteria, which together count toward 2.5% of the total points possible. Alfresco can bump up their score by extending current XForms functionality to the DM side of the house which is long overdue. Email archiving should get a better rating when the new “email addressable objects” functionality is released.

This starts a theme that repeats itself throughout the report: In keeping with the “Suite” approach, Forrester expects an ECM vendor to offer better E-forms and Email archiving, for example, than pure-play vendors. This automatically favors the largest of the ECM vendors because they simply acquire and integrate pure-play offerings. So don’t expect a “5” in these areas for Alfresco any time soon using the “Suite” standard.

“Core capability architecture” scored well, which is a measure of Alfresco’s ability to work in an “enterprise” IT environment, but, strangely, this score is only worth 5% of the total. That’s not a typo. The ability for the product to leverage a client’s application and security infrastructure as well as its extensibility is not as important, in Forrester’s view, as the ability of the ECM vendor to offer Business Intelligence software (see “Differentiation Strategy”).

Forrester’s expectations for the breadth of an ECM Suite become clear when you look at what a vendor would have to do to get a perfect score. And this is where I start to question the value of their definition of the ECM Suite, let a lone the scorecard itself. Let’s look at a few examples…

Under “Core ECM capability breadth”, the “Content Integration” category is interesting. On the face of it, I would think products that support open standards and have many API’s available for integrating the repository with other systems would do well here. On the contrary, Forrester gave Alfresco a 1.0. Here’s what a vendor must offer to get a 5:

“Comprehensive, independent content integration offering supporting central, federated bi-directional access and management of content in heterogeneous, disparate repositories and content information sources.”

It’s like the “E” in ECM stands for “Excessive”. Seriously, I’m sure certain clients will have a legitimate need for this functionality but there are more common, more practical integration-related concerns vendors should be striving for.

Let’s turn our attention to “Extended capabilities” which is worth 10% of the total points possible. Alfresco got an overall score of 1.05. Alfresco’s lack of an Enterprise Search offering (Score of 0 with a weight of 15% for the category) and, in my opinion, a mis-rated BPM capability (Score of 1.0 with a weight of 20% for the category) were the primary culprits here.

The inclusion of Enterprise Seach as part of an ECM Suite evaluation floors me. Does anyone really want to buy an enterprise search engine from their ECM vendor? I suppose if you are IBM you are happy it is included. It is about this point in the report I start to form a mental image of Forrester’s Dream ECM Suite as the giant, lumbering Imperial Star Destroyer shown in the opening sequence of Star Wars (or, better yet, the parody of the same in Spaceballs in which the enormous ship has a bumper sticker that reads, “We Brake for Nobody”).

The BPM score is simply a bad rating. Either the criteria for BPM is so overblown as to not be relevant to the majority of clients who need workflow within an ECM suite or the analysts simply did not do enough research into the capabilities of the embedded JBoss jBPM engine. Forrester doesn’t offer any details as to why it considers the functionality to be “basic” so we’re forced to guess.

Collaboration is another example where Forrester’s threshold is set way too high. Alfresco scored a 2. To get a “3” in this criteria–an average score–you have to offer real-time web conferencing. Huh?

When you get to “Strategy” and “Market Presence” you realize who Forrester values: Huge vendors that offer everything to everybody and have more than $10 billion in annual revenue (that’s what it takes to earn a 5). And to be successful, the vendor must not focus exclusively on ECM. Take a look at “Differentiation strategy”. This one’s important to Forrester. At 12.5% of the total score it represents the single most heavily weighted item in the scorecard. To score the max, here’s what a vendor has to do:

“Vendor differentiates by offering ECM as part of a broader information management portfolio (data, master data management, BI, content, BPM, portal, and collaboration technologies) with a promise of lower TCO.”

Hmm. Who could offer broader information management, Business Intelligence apps, Portal, AND collaboration technologies? Wait, don’t answer yet. The vendor musn’t stop there. It’s not quite broad enough. To score max points on “Vendor’s business”, another 5% of the total, the vendor must provide “…IT application infrastructure solutions including application infrastructure, developer tools, middleware, and information management.” Basically, if your name is IBM, Oracle, or Microsoft, give yourself a 5.

So Alfresco took a couple of knocks here and there it probably shouldn’t have. The bigger issue is that Forrester’s vision of ECM is out of step with what clients want and their scorecard reflects it. No one’s in the market for Imperial Star Destroyers anymore.

I’m back from the Alfresco Community Conference which was held Wednesday in New York City. It was a day well spent with excellent presentations given by John Powell, John Newton, Kevin Cochrane, Paul Holmes-Higgin, and Helen Dann. Christian Science Monitor’s Russ Danner gave an update on regional Alfresco user groups. And three Alfresco clients–Kaplan, Harvard Business School Publishing, and Film Solutions–all gave brief updates on solutions they’ve built with Alfresco. It sounds like Kaplan has a significant Alfresco WCM-based site going live this week, BTW.

The conference also gave me the chance to meet several of you in person. Thanks for the kind words and your continued interest in the blog. I made some progress on the Advanced Workflow article on the plane home so stay tuned for that.

Here are some (rough) notes from the conference…

Alfresco has released a new independent benchmark

• 100 million objects
• Loaded the objects at 140 objects/second (using an RMI interface on a different node than the repository)
• Unlike some competitors–ahem, Microsoft–there are no artificial limits in terms of the number of objects in a collection or the number of collections.
• Linear scalability–there was no degradation as volume increased.
• Sub-second response times.
• Next stop is 1 billion objects. An impressive side note is to compare how long it took Documentum to get to a billion objects (~10 years?) with Alfresco’s efforts (~3 years?).

Alfresco’s architectural principles (para-phrased):

• Content is a service
• Everything must be modular
• Lightweight scripting is important
• Enterprise scale
• Must fit into the user’s environment (multiple platforms, multiple browsers, multiple databases, etc.)
• The web is the interface
• Continuous innovation

Problems with Sharepoint (I’m considering doing a Sharepoint vs. Alfresco smackdown article so we can get a real independent comparison going, but for now, here’s Alfresco’s opinion):

• Slow, doesn’t scale. (Limited to 5 million docs per library and about 10 libraries total).
• Hard to program and extend.
• Poor WCM.
• 100% Microsoft stack from browser to back-end and everything in-between.
• John Newton said, “IBM should be competing directly with Sharepoint but they aren’t providing any guidance.”

A comment John made that is begging to be drilled into is that he considers Alfresco’s Office integration to be better than Microsoft’s. This is a bold claim. Most people would cite “deep MS Office integration” as one of the things that would immediately take you down the Sharepoint road. So I guess I’ve got a to-do to figure out how much truth there is to that claim.

Exciting things just released or coming soon:

• Major commitment to Flex
• Flex SDK
• New Flex-based client for Knowledge Workers called “Networks”
• Joomla, WordPress, and MediaWiki integration
• Facebook and iGoogle integration. Essentially run facebook apps/gadgets that expose the Alfresco repository. It sounds like Alfresco will also be an OpenSocial container when OpenSocial is ready.
• JSR-283 which provides a SQL-like query capability to the repository (Recovering Documentum developers, think “DQL”).
• Support for ATOM Publishing

What’s going on with various releases (None of these dates are necessarily set in stone. Check the Alfresco wiki for the official roadmap):

• 1.4 is nearing end of life. Everyone should be upgrading to 2.1.1.
• 2.1.1 is now available.
• 2.2 is on its way and should be out before the end of the year.
• 2.9 Community (Q4 of 2007)
• 3.0

• Community in February of 2008, Enterprise in Q2
• WYSIWYG for mash-up compositions
• Flex-based RIA client for ECM

More info on 2.2:

• This release is the foundation for the new collaboration features that will start to find their way into the product.
• Email addressable spaces. Allows every object in the repository to have an email address. For example, mail a piece of content to a folder in the repository. Respond to an email about a piece of content and start a discussion thread that gets persisted to the repository.
• Branching. Allows web projects to start as copies of an existing web project.
• Web Content ACL’s. Exposes object level permissions in WCM to the UI. (Previously, only the DM store could be used to assign permissions at the object level. The WCM store was limited to the site level).
• In-context editing. Allows someone to click an icon while previewing a web site that immediately launches the web form used to create that content.
• Find content created with a specific web form. Adds a link to the sandbox page next to the “Create Content” link for each web form that takes you to the content created with that web form. MUCH NEEDED.

More info on 2.9 community:

• Like a preview of some significant features that are coming in 3.0
• Dynamic data dictionary. Allows you to implement new models or override existing models without a restart. YES!
• Multi-tenancy. Good for internal or third-party hosting situations in which you want to essentially partition the repository so that two or more “tenants” would never know the others existed. Segregates workflows, repositories, models, etc.
• iGoogle/Facebook integration. Exposes things like search, workflow tasks, and “document library”.
• Collaboration AMP. Includes ability to post content to WordPress, use Alfresco for the back-end to MediaWiki, IM presence integration.

All in all a good opportunity to hear from and interact with Alfresco leadership and others in the community. I’m looking forward to the next one.