Alfresco Maven SDK: Declaring other AMPs as dependencies

5634268846_9c59682a42_mI’ve seen a few questions about how to configure an Alfresco project to depend on other add-ons when using the Alfresco Maven SDK, so I thought I’d do a quick write-up on that.

When you use the Alfresco Maven SDK Archetypes to bootstrap a project, you can select one of three project types:

  1. alfresco-allinone-archetype: Referred to simply as “All-in-One”, this type, as the name implies, gives you a project that facilitates creating both a repository tier AMP and a Share tier AMP, plus it includes a working Solr installation.
  2. alfresco-amp-archetype: This sets up a project that generates only a repository tier AMP.
  3. share-amp-archetype: This sets up a project that generates only a Share tier AMP.

In my Alfresco Maven SDK Tutorial I use both the alfresco-amp-archetype and the share-amp-archetype but I don’t really talk about the All-in-One archetype at all. The reason is two-fold. First, I think for most beginners, the smaller, more focused archetypes are less confusing. Second, on the vast majority of my client projects, that’s the setup I use. Usually, I feel like the All-in-One archetype is overkill.

However, there is an advantage the All-in-One archetype has over the other two. It is able to pull in other AMPs as dependencies. With the single-purpose archetypes, your goal is simply to build an AMP. You can run integration tests against that AMP using the embedded Tomcat server, but if you want to test the AMP with other AMPs, you have to deal with that manually. Usually I just deploy the AMPs to a test Alfresco installation that already has the other AMPs deployed.

But with the All-in-One archetype, you can declare those other AMPs as dependencies, and Maven will go grab them from a repository and install them into your local test environment using something called an “overlay”.

Here’s an example. Suppose you are developing some customizations that you know will be deployed to an Alfresco server that will be running the following add-ons: Share Site Creators, Share Site Space Templates, Share Login Announcements, and Share Inbound Calendar Invites. You’d like to test your code alongside these add-ons.

One approach would be to go grab the source for each of these projects and build them locally to produce their AMPs (or download pre-built AMPs), then deploy all of those AMPs to a test server, then deploy your custom AMP and test.

But all of those add-ons happen to live on a public Maven artifact repository called Maven Central. So a better alternative might be to simply list those modules as dependencies and let Maven take care of the rest.

Here’s how it works. When you generate an all-in-one project, the structure includes not only folders that contain your repo and Share AMP source, but also folders representing the Alfresco and Share web applications. For example, here are the files and folders in the root folder of a project called “test-allinone-220”:

pom.xml
repo
run.bat
run.sh
runner
share
solr-config
test-allinone-220-repo-amp
test-allinone-220-share-amp

In our example, your custom code for the repo tier would go in test-allinone-220-repo-amp and your Share tier code would go in test-allinone-220-share-amp.

The repo and share directories are used to build the respective WAR files and they each have their own pom.xml. So to add the other add-ons to our setup, first edit repo/pom.xml. You need to add the add-ons as dependencies, like:

<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>inbound-invites-repo</artifactId>
    <version>1.1.0</version>
    <type>amp</type>
</dependency>
<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-space-templates-repo</artifactId>
    <version>1.1.2</version>
    <type>amp</type>
</dependency>
<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>share-login-ann-repo</artifactId>
    <version>0.0.2</version>
    <type>amp</type>
</dependency>
<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-creators-repo</artifactId>
    <version>0.0.5</version>
    <type>amp</type>
</dependency>

Then, add them again to the overlays section (without the “version” element), like this:

<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>inbound-invites-repo</artifactId>
    <type>amp</type>
</overlay>
<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-space-templates-repo</artifactId>
    <type>amp</type>
</overlay>
<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>share-login-ann-repo</artifactId>
    <type>amp</type>
</overlay>
<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-creators-repo</artifactId>
    <type>amp</type>
</overlay>

That covers the Alfresco WAR. Now for the Share WAR, edit share/pom.xml and add the Share tier dependencies, like:

<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>share-login-ann-share</artifactId>
    <version>0.0.2</version>
    <type>amp</type>
</dependency>
<dependency>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-creators-share</artifactId>
    <version>0.0.5</version>
    <type>amp</type>
</dependency>

And then the overlays:

<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>share-login-ann-share</artifactId>
    <type>amp</type>
</overlay>
<overlay>
    <groupId>com.metaversant</groupId>
    <artifactId>share-site-creators-share</artifactId>
    <type>amp</type>
</overlay>

Now you can run the whole thing–your custom AMPs plus all of the dependencies–by running ./run.sh. As part of the build, you should see the dependencies being downloaded from Maven Central. As the server starts up, if you watch the log, you’ll see the modules get initialized. And when you log in, you’ll be able to test all of the functionality together.

When you are ready to deploy to one of your actual server environments, you have a choice. You can either copy all of your AMPs to the amps and amps_share directories, then run bin/apply_amps.sh. Or you can deploy the WAR files that are now sitting under repo/target and share/target. Personally, I think re-applying the AMPs against a fresh WAR on the target server is less confusing and less error-prone, but I’ve seen it done both ways.

If the add-ons you want to depend on are not on a public Maven artifact repository like Maven Central, you can still build them locally, run “mvn install” to install them into your local Maven repository, then depend on them as shown.

Activiti founders fork the project to create Flowable, an open source BPM engine

flowable-with-taglineOn Thursday the news broke that Activiti had been forked to create a new open source Business Process Management (BPM) engine called Flowable. Activiti is a BPM engine that Alfresco Software, Inc. funded to replace JBoss jBPM. The Activiti engine can automate business processes as a standalone application, but it is also the embedded workflow engine that Alfresco ships as part of its ECM platform.

A fork is when the evolution of an open source project intentionally diverges. Instead of one code base controlled by a single entity, a fork creates a second code base from the first, allowing each project to evolve separately going forward in accordance with their own distinct roadmap and governance model.

Forks happen fairly frequently, and in places like GitHub, where a lot of collaboration around open source software takes place, they happen constantly, by design. But when a significant fork happens to a project, especially one that has commercial backing, like this one, it is worth taking note. Alfresco depends on Activiti for its ECM platform, so now it will be forced to either continue to develop Activiti under their own direction, or switch to the new project like any other upstream dependency.

The news of the fork comes about a month after my blog post reporting that multiple Activiti engineers had resigned from Alfresco. It is now public that those engineers were Joram Barrez and Tijs Rademakers, who were absolutely the heart and soul of the Activiti project. Those departures left a big question mark hanging over the future of Activiti.

Then the big news came on Thursday. Tijs Rademakers released this tweet, announcing a new project, Flowable, with a link to a blog post.

tijs-tweet

The blog post outlined some high-level reasons for the fork:

“We acknowledge Alfresco’s stewardship of the Activiti.org project, and as employees we enjoyed considerable freedom to develop the project over several years. However, things didn’t work out as we expected or hoped. We came to the conclusion the only way to continue evolving our ideas was to fork.”

The blog post goes on to talk about forks, in general, and their potential impact.

I wanted more detail so I spent some time talking to members of the new Flowable project last week to understand why they forked Activiti and what we can expect going forward.

The first thing that is immediately obvious when talking to any of the former Activiti engineers is how much passion they have for the project. Leaving Alfresco was clearly a hard decision. But the hardest part was feeling disconnected from the software they created and the fear that the project would no longer have the stewardship it enjoyed prior to their departure. “We couldn’t just leave it there,” said one of the engineers. Indeed, looking at the Activiti forum as well as the commits for Activiti on GitHub, it is obvious that Alfresco is still working to rebuild the Activiti team.

Activiti Commits

After their departure, the former employees continued to discuss with Alfresco Software how they might be able to move forward, but the two sides could not reach an agreement. It soon became obvious to the team that in order to move the project forward they would have to take more direct control, and the only way to do that was to fork.

Specifically, the team started with the latest Activiti release, 5.21.0, and added a new feature called “transient variables” and some other small enhancements as well as bug fixes, and released that as Flowable 5.22.0 (GitHub).

It’s actually not unlike the creation of MariaDB after Oracle acquired Sun (and then became owners of MySQL). In fact, it is the stated goal of the engineers for Flowable to be a drop-in replacement for Activiti, just like MariaDB is for MySQL.

The engineers say they are looking forward to getting back to their open source roots. “In true open source fashion, we want people to get involved in whatever way they can, whether that’s documentation, testing, or code contributions,” said the team. The Flowable project will run as a meritocracy, with trusted developers earning commit rights over time, regardless of their commercial affiliation.

The fork has actually made many people in the community a lot more comfortable with the direction of the project because the engineers who created the software and made the vast majority of the commits during its life are now committed to moving it forward, and this fork gives them the freedom to do it. “Everyone has been supportive so far,” the team said, “Now the burden is on us to start releasing.”

Clearly, Alfresco Software is unhappy with the fork. In a blog post published Friday, John Newton, Alfresco co-founder and CTO, said he was hoping for a different outcome:

“Unfortunately, some of my early friends on the Activiti project have disagreed with our direction and have taken the step of forking the Activiti code. This is disappointing, because the work that they have done is very good and has generally been in the spirit of open source. However, the one thing that we could not continue to give them was exclusive control of the project. I truly wish that we could have found a way to work with them within a community framework.”

John’s post indicates that the company intends to continue to develop Activiti. John writes, “We are redoubling our determination to grow Activiti faster, better and more openly.”

Growing Activiti faster will be key. If Alfresco is slow to field a new Activiti team and then fails to give the market what it wants at the pace it wants it, those stuck on a lagging codebase may become frustrated, particularly if Flowable is able to out-innovate Alfresco. Flowable has the advantage here because they are the original core team, they are more nimble, and they can hit the ground running. But without Alfresco as a sponsor, Flowable engineers must find a way to earn a living as well as move the project forward. Ideally they will be able to make the two intersect by finding clients to pay them to stay focused on the project.

Alfresco’s product and strategy depends heavily on Activiti. From a purely commercial perspective it is understandable they would want as much control over the codebase as possible. However, there are other commercial open source companies that have strategic dependencies on upstream open source projects they don’t fully control. Elastic’s dependency on Lucene comes to mind. If Alfresco could participate in the Flowable project in a similar way it could potentially benefit everyone, but John’s post seems to indicate that is not in the cards. The next few months will tell us just how committed the company is to continuing with Activiti as a separate project.

Personally, regardless of what happens with Activiti, I hope Flowable flourishes. If team member passion was the only predictor of success I’d say it’s a sure bet it will.

Keep your users informed with this new Alfresco Share add-on

A common request from clients is the ability to manage a simple set of system announcements which can be displayed on the Alfresco Share login page. These might be used to let everyone know about important events like an upcoming system outage or that there are cronuts in the breakroom.

This is a straightforwaShare Announcementsrd add-on to write, but I thought it might be useful to others, so I’ve packaged it as an open source add-on available on GitHub.

The idea is simple: When you install the module, it will automatically create a folder in the Data Dictionary called “Announcements”. Administrators will create one plain text file in that folder for every announcement. There is no custom content model or properties to mess with. The content of the file is the body of the announcement.

An Alfresco Share Extension Module does the work on the Share side. When a user opens the Share login page, a Share-tier web script calls a new announcements web script on the repo tier that returns the announcements as JSON. The Share extension formats the JSON response as an un-ordered list that gets injected into the login page between the Alfresco logo and the username/password fields.

When you are ready to take an announcement down an admin can either delete the announcement file or move the file to an arbitrary folder under “Announcements” to save for later reuse.

Some installations–like those with Single Sign-On enabled–never see the Alfresco login screen. For those installations, or for anyone that wants to expose the announcements to logged in users, the add-on also includes an announcements dashlet.

Announcements DashletHopefully you’ll find this useful. If you find issues or want to make enhancements, pull requests are always welcome.

Key Activiti engineering leads have resigned from Alfresco Software

activiti_logoIndependent sources within the company confirm that key leaders of the Activiti team have just resigned from Alfresco Software. Despite rumors to the contrary, this was not a coordinated effort with the team leaving en masse–each of the three departing engineers are doing so independently, for their own reasons.

Activiti is an open source Business Process Management (BPM) system that can run independently as a standalone workflow engine. Alfresco embeds Activiti in its Enterprise Content Management (ECM) platform and offers commercial support for standalone Activiti under a product offering called “Alfresco Activiti”.

Whether it is run within Alfresco or as a standalone BPM engine, Activiti is a key technology that enables businesses to streamline processes that involve both humans and other enterprise systems. The departures are awful news for the company, which has been doing a lot of marketing around automating business processes in recent months.

Alfresco created Activiti as an open source project to replace jBPM, which, at the time, did not support the BPMN specification and was unwilling to modify their license to be compatible with Alfresco’s. Alfresco hired the jBPM creator and several members of that engineering team to create Activiti as a new BPMN-compliant, Apache-licensed engine from the ground, up.

The project has been popular and successful, both in its open source community and as a commercial offering, despite hitting a bump in the road in 2013 when some of the community members decided to fork the project.

What is clear is that the Activiti open source project will continue to move forward. The most likely scenario is that one or more of the departing engineers will continue to contribute to Activiti–they’ll just earn their income from another source. I suspect that Alfresco will continue its commercial push behind BPM and will continue to leverage Activiti as it does hundreds of other open source components.

A company always suffers for a while when a good person leaves. In this case several good people are leaving. I hope Alfresco can deal with this change and move forward successfully, and I’m looking forward to seeing what these talented engineers do next.

Alfresco embraces AngularJS: Now what?

Alfresco plus Angular2In 2009 I wrote a blog post called “Alfresco User Interface: What are my options?”  because people frequently asked for recommendations around that and they still do. Then, in 2010 at the very first Alfresco DevCon, I gave a talk called “Alfresco from an agile framework perspective“. My talk summarized how painful it was to build custom apps on Alfresco by building one simple app twice–once using Share and once using Django–and then comparing the effort it took. Django won, hands down, in terms of level-of-effort and lines of code.

Share was fairly new at the time, and it has improved a lot since 2010 in terms of ease-of-customization. In fact, a few of us met with Alfresco engineering at the conference to give feedback about the Share developer experience and Alfresco listened. The next release of Share had improved extension points. It became much easier to extend or override pieces of Share without massive copying-and-pasting of Alfresco’s code.

But there was another aspect of my talk, which boils down to this: If you are trying to appeal to developers, why are you asking us to learn a new framework? There are already frameworks out there that are extremely popular, have all sorts of tooling, and are well-documented. Use what you want to build Alfresco Share, but what can you do to make it easier for developers to build content-centric applications on Alfresco using tools we already know?

Fast-forward to BeeCon, the community-led Alfresco conference held in Brussels back in April of this year. During that conference, Alfresco announced that it was doing something about this problem: They would create libraries and components to help people create custom apps on top of the platform external to Share. Some of these will be framework agnostic. But after looking at the frameworks out there they realized they cannot deny the popularity of AngularJS, so they also want to provide customizable components that Angular developers can use.

About two weeks ago, we got our first glimpse of what this really means. Alfresco made its alfresco-ng2-components GitHub repo public followed closely by a live streamed hangout where they discussed the vision for the components (YouTube link).

Definitely clone the repo, try it out yourself, and watch that video for context. But let me try to summarize what it means for you…

What is happening to Alfresco Share, Aikau, and Surf?

Alfresco Share is the shipping web client. It is built on top of Aikau and a lower-level framework called Surf. If Alfresco is getting behind Angular, are these going away? No. Both John Newton, Alfresco’s co-founder, and John Sotiropoulos, Alfresco’s new VP of Applications, were emphatic on that point. There are no immediate plans to make any significant changes to the direction of Alfresco Share, Aikau, or Surf. Many people have customized Share–those customizations should remain safe. Plus, a lot of people use Share out-of-the-box. Alfresco still needs a web client and they are their own biggest fans when it comes to Aikau and Surf.

I am customizing Share right now. Should I stop?

No, these components aren’t going to affect you. If you have no reason to build a custom front-end on top of Alfresco, you can safely ignore this announcement. However, if you are customizing Share because you thought it would be easier than writing your own custom application, and your customizations are so significant, that it feels like you are basically re-building an application from the ground-up, with little or no re-use from what Share offers, you may want to keep an eye on this. These components will give you a jumpstart on what otherwise would have been a from-scratch effort.

What exactly is shipping right now?

Alfresco is making an initial set of components available that includes:

  • Login
  • Document list
  • Document viewer
  • Datatable
  • Search
  • Uploader

In addition to the components, Alfresco offers a Yeoman generator to help you bootstrap an Angular2 application that makes use of one or more of these components.

I am writing a custom front-end using Angular right now. Should I stop?

Alfresco is shipping Angular components that can be used to build custom apps on top of Alfresco. And you are doing the same thing. It would seem like you might want to stop and leverage the new components. Before you do that, you need to be aware of two very important caveats.

First, Alfresco is using Angular2 which has very significant differences with Angular. And Angular2 is still shipping as a release candidate. Depending on your appetite for refactoring and risk, you may or may not want to make the switch to Angular2 just yet. Still, I think it was smart for Alfresco to choose Angular2 to avoid making that move later.

The second thing is that the new Alfresco Angular2 components rely on significant enhancements to the Alfresco REST API. Those enhancements are not in any stable version of the product. They are only available starting with 201606 EA (Early Access) release, which includes 5.2.a of the Alfresco platform. There is no way to know when Alfresco 5.2 Community Edition will go to a GA (Generally Available, aka “stable”) release. Which means we also do not know when 5.2 Enterprise Edition will ship.

You should not run any Early Access release of Community Edition in production. Therefore, if you need to be in production any time soon, you will not be able to leverage these components.

This second caveat has caused a fair amount of grumbling amongst the community. There has been some talk about back-porting the new REST API to a stable Alfresco release, but, unfortunately, this cannot be done without changes to the core.

People have been developing custom front-ends on top of Alfresco using Angular and other frameworks for years without making deep changes to the core product. Alfresco didn’t have to do it in this case, but they chose to, because they wanted to clean up the API simultaneously. Basically they chose the “rip the bandage fast” approach, which I understand. However, the impact of this decision is that it will be many, many months before real world feedback on these components makes it into the shipping code. Hopefully, enough people will still be motivated enough to try them out with the EA release.

I want to try the new components, how do I get started?

First, you’ll need a running version of 5.2.a (201606-EA) with CORS enabled. I forked the gui81/alfresco Docker image and upgraded it to 5.2.a with CORS enabled and pushed the result to Docker Hub if you want to use it. Or you can download it from the wiki or Sourceforge.

The GitHub repo has a good readme, so take a look at that. I’m not going to duplicate it here, so follow those directions.

I had good luck installing the Yeoman generator and then just running “yo ng2-alfresco-app”. You’ll get a chance to specify which components you want installed into your new app–I had problems unless I selected all of them.

Yeoman generates the app. NPM is used to run it locally for development purposes. To fire it up, just run “npm start”. Once up, I was able to use the sample app to work with the back-end Alfresco repository, as shown here (click to enlarge):

Alfresco Angular2 Components

It’s probably important to note that what you see running is basically a sample app or sandbox for these components. It’s up to you to write a functional app with all of the features your users need. You’ll incorporate and customize these components as part of your own development effort. You can check the documentation to learn more about how to customize each component.

Alfresco wants your feedback on these components. Don’t be afraid to report issues or submit pull requests on the GitHub repo.

Summary

I am excited to see this from Alfresco. It is too early to use these in production, but once the new REST changes ship in a GA product, you should definitely consider this as a way to jumpstart your custom apps, even if you don’t use Angular. If you’ve been looking for a way to get involved in the Alfresco community, trying these out and giving your feedback is an excellent opportunity to do so.

An open letter to my son on the eve of starting his first summer internship

IMG_1086Well, Justin, high school graduation is behind you, you’ve landed in San Jose, and tomorrow morning you’ll wake up in Mountain View to start your internship at Mozilla, your first real world, paying job that’s aligned with your career goals.

I am so proud of you for all that you’ve accomplished and I know you’re going to have a great summer. But I have this fatherly need to impart wisdom, and I couldn’t fit it all in at the airport drop-off this morning, so I’m putting it here and linking to it from Twitter where you’ll be more likely to read and absorb it…

1. Don’t stand around with your hands in your pockets.

In every business there is always work that needs to be done. Don’t literally or figuratively just stand there doing nothing. Jump in and contribute value.

2. There is something to learn in even seemingly crappy tasks.

Every project has less desirable tasks and someone’s got to do them. If that’s you, don’t get discouraged. Take the grunt tasks on with a smile, knock them out quickly with quality, and find a lesson those tasks offer. Learning something new makes any task worth it.

3. Learn something new every day.

Speaking of learning, try to learn something new, no matter how small, every day. Actually, this applies to the rest of your life, not just your work life. If you get to the end of the day and you can’t answer, “What did I learn today?” that’s a day wasted, and it’s on you to fix.

4. Be confident, but practice gratitude and humility.

You’ve contributed countless hours to Mozilla, you’ve honed your skills, and you’ve successfully navigated the vetting process. So be confident that you’ve earned the right to be there. But something to reflect on daily, maybe as you walk to work in that gorgeous northern California weather, is how fortunate you are to have this day, this opportunity, in this place, and what you will do now to capitalize on it.

5. Have fun. Live in the moment.

This summer is going to fly by. And you’ll have your whole life to work. So my biggest piece of advice is to have fun, try new things, and to savor every single moment. Seriously, it sounds easy but it is so hard to do. When you are with your friends, standing on that hill overlooking the Golden Gate Bridge, I hope you’ll pause, take a few breaths, and really just soak it up.

So there you go. Hopefully, I’ve been working these in to your brain over the last 18 years so maybe it didn’t need to be said, but thanks for letting me say it anyway.

Oh damn, I almost forgot: Dress in layers, Facetime your Mom, Snapchat your sister, keep your wallet in your front pocket on public transit, if you’re on-time you’re late, be discreet and watchful at ATMs, always wear a helmet, be a thoughtful and considerate roommate, and stay away from the Tenderloin.

Why Alfresco follows the same strategy as closed source commercial software companies

soundwalk_by_finishing-schoolMy friend and colleague, Peter Löfgren, recently wrote a blog post on what he sees as the two possible strategies Alfresco could pursue. He describes the two approaches as follows:

Vertical: You try to get as many as possible Community installs be converted to paying Enterprise customers. Convert from the bottom and up if you so like.

Horizontal: You try to get as many as possible to run Alfresco, even if they run the free Community version. A fair share will always want commercial support to have a professional backing. A broad approach, where you get market presence and self-sustained marketing.

Peter argues that, to date, Alfresco has used a vertical approach, which is really about pushing Enterprise Edition and begrudgingly acknowledging that Community Edition is an acceptable alternative, only when the client can’t currently justify the expense of an Enterprise Edition license.

Peter observes that the vertical approach pits Alfresco Enterprise Edition squarely against Community Edition making it very tempting for Alfresco sales people to badmouth Community Edition, because they often see it as cannibalizing their revenue. I actually don’t see this happening much anymore–sales people know that denigrating Community Edition is counter-productive because customers are savvy enough to know it is the same software and a good portion of the sales team understands the bigger picture.

Before I go any further, I should refer you to a blog post I wrote about a year ago called, “The plain truth about Alfresco’s open source ethos“. In it, I argue that Alfresco’s marketing strategy isn’t the community’s concern, and that the company is basically a “normal” software company that won’t ever be the dogmatic open source company many of us wish it was.

But Peter brought it up and I like discussing such things, so I’ll ignore my own advice and provide my take on why I think Alfresco will continue to ignore the horizontal strategy and will continue to basically act like any other traditional software company…

Here’s my take

First, let’s compare Alfresco with another commercial open source company, Elastic. Elastic is the company behind the popular search engine Elasticsearch as well as a variety of related big data and analytics tools such as Logstash, Kibana, and Beats. Like Alfresco, Elastic makes money from support, and their incentive to get people to pay for support is to offer a set of products they make available only to paying customers. Unlike Alfresco, Elastic ships a single distribution of their products. For a given release of Elasticsearch, for example, there is no difference between what a paying and non-paying customer downloads and runs. It’s just that if you want some additional value-add on top of what’s freely-available, you have to pay.

So this is an example of a horizontal pursuit as Peter describes it. The reason it is working for Elastic, though, is that their offerings are much more horizontal than Alfresco’s. Elasticsearch is more popular and more widely used in all kinds of use cases. Their download stats are impressive and increasing steadily.

Alfresco, on the other hand, is niche software. It is quite narrowly focused on document management. Yes, there are a lot of use cases within that, but it isn’t something that you see embedded in all sorts of applications like you would a database, a workflow engine, or a search engine. I suspect download stats are flat or maybe even decreasing, although this is a bit of an apples/oranges comparison as the two products are in different phases of maturity and adoption.

The other issue is one of leadership. Elastic’s CEO, Steve Schuurman, exudes open source. He was a co-founder of SpringSource, for example. Both he and Shay Bannon, the creator of Elasticsearch, have said repeatedly that Elastic will always be open source and that they’ll never have an Enterprise-only distribution of their core software. That’s the kind of leadership I expect from a commercial open source company.

Contrast that with the current leadership at Alfresco. When former CEO John Powell announced his retirement, the board could have chosen someone with open source credibility like Elastic’s Steve Schuurman. Instead, they went with Doug Dennerline. He and his lieutenants have next to zero open source credibility or experience. It is clear they were brought on solely to take the company public. For them, open source is not a driving part of their worldview. Instead, their focus is simply to build a software company and take it public, employing whatever strategy gives them and their shareholders the biggest revenue numbers year-after-year. (I don’t mean to paint this in an overly-negative light–it is what it is. I’m just trying to point out the stark contrast in motivation and philosophy between the two leadership teams).

Unfortunately, a horizontal strategy does not necessarily equate to those kind of numbers. With Red Hat as the notable exception, it is hard to find a commercial open source company with financials that Doug, the board, and investors are looking for.

Do the math. Let’s assume there are 50,000 installs of Alfresco Community Edition. I have no reason to believe that is accurate–this is just an exercise. What kind of conversion rate would you expect? You’ll probably guess too high, forgetting that Alfresco is now very expensive, even for modest installations, and that the company is still working to add more differentiation in its paid offering compared to the free product. Let’s use 2%. So that’s 1,000 paying customers, which is roughly the number John Newton disclosed publicly in a keynote several years ago. It’s probably higher now, but remember that there is attrition we haven’t accounted for and those customers have to be earned year after year.

Now, what do you think the average sales price of Alfresco is across all of their paying customers? Again, just spit-balling, let’s say it is $100k annually. Multiply that times 1,000 and that’s “only” a company with $100 million in annual revenue. If you’re looking for a $1 billion IPO, that’s not enough. (If EMC sells Documentum to someone for 10x revenue I’ll have to update this post, but I think I’m pretty safe).

In a horizontal strategy, those are your levers: Total installs, conversion rate, attrition, average sales price. For example, using the horizontal approach, to increase revenue from $100 million to $300 million you would have to triple the number of Community Edition installs from 50,000 to 150,000. Alternatively, you could keep CE installs steady at 50,000 and instead triple your conversion rate from 2% to 6%. Which seems easier?

My bet is that rather than increasing total Community Edition installs, Alfresco will find it easier to increase the conversion rate by increasing differentiation between the two products, cutting attrition by implementing “customer success programs” and consulting, and continuing to put upward pressure on the average sales price by charging more for the core product and finding new paid add-ons to sell.

The horizontal approach Peter advocates may be the one we all wish would work, but I think that ship has sailed.

Celebrating more than 10 years covering open source ECM on ecmarchitect.com

3728891766_ec7def01b7_mI started blogging back in 2001, stopped, then started again in 2002. Those early posts were all over the map, topic-wise, and they were often very short blurbs on really random stuff. I was scratching a creative itch–I just wanted to write and I didn’t care too much about what. It wasn’t until 2005, when Alfresco was first released, that I started to narrow my focus and really find my voice.

I’m often uncomfortable singing my own praises, but over the last ten plus years I’ve heard from so many of you–in person, via email, in comments, and forums–that what I’ve written has made a significant difference in your professional lives. For some it simply helped you fix an annoying issue. For others it saved a project. And multiple people have credited it with giving them the confidence to make a career change into jobs where they could follow their passion. Regardless of the significance, each of those stories makes me very happy.

I started this blog selfishly, as a way of keeping track of what I was learning about open source ECM, search, and workflow, and documenting that for my teammates and clients. I figured why not make the knowledge available to the public–maybe a few others outside of those circles would derive benefit from it.

What I didn’t know at the time was that my blog would eventually:

  • Lead to a new job at what was then an open source-focused consulting firm
  • Result in a book deal and then another book deal
  • Launch a temporary career experiment in community leadership in a C-level position
  • Become a go-to reference for half a million technologists and business users from all over the world (true, that’s not “unique users”, but still).

I realize I’m not solving world peace. And this is one of many tech blogs focused on ECM and related technologies. But it’s kind of cool to see what ten years of picking a subject and writing about it when the mood strikes me ultimately turned into. So I wanted to take a minute to kind of appreciate that.

I also wanted to say thanks to you. To those reading this now, to anyone that’s ever left a comment, to anyone that’s ever stopped me and told me your story: Seriously, thank you. Your continued support means a lot.

What’s next for ecmarchitect.com? No idea. I’m sure you’ve noticed that I’ve been trying to mix it up a bit, to be less Alfresco-focused. That’s reflected in my client work, so it’s natural it comes through here as well. Ultimately, I suppose I’ll continue to write about what interests me, and hopefully you’ll continue to find it interesting and helpful as well.

If you have ideas on topics you think might be good to explore in this space, let me know in the comments.

What’s new in Alfresco Community Edition 5.1?

This post covers the significant new features of Alfresco Community Edition 5.1. I’ve also published a YouTube video that demos the new features.

New Release Names

Alfresco Community Edition 5.1, which you may see referenced as “201602-GA”, has been released. This is the first GA release of Alfresco Community Edition using the new release naming nomenclature. Let’s talk about that first, then I’ll give you the highlights of what’s new.

In the past, Community Edition releases were assigned letters. For example, the last release in the 5.0 line was 5.0.d. Historically, Alfresco would give no clues as to whether or not they considered that a stable or final release. Subsequently, people would just grab the latest release and install it, which led to all sorts of problems. (See “Alfresco Community Edition needs sensible version labels“).

Now, thankfully, Community Edition releases are either Early Access (EA) releases or Generally Available (GA) releases. The EA releases are essentially snapshot releases that are stable enough for the community to try out and provide feedback on. The GA releases are stable builds. If you are running Community Edition in production, you should be running GA releases, not EA releases.

So the release we are talking about is “201602-GA” which means it was released in February of 2016 and it is GA. I don’t know what they are going to do if they have two GA releases in the same month, but I guess we’ll deal with that if it happens. Given that GA releases do undergo testing by Alfresco QA, which can take some time, two releases within the same month may be unlikely.

Alfresco is actually a collection of components. Community Edition 201602GA contains Alfresco Platform 5.1.e, Alfresco Share 5.1.e, Aikau 1.0.39.5, AOS 1.1, and Google Docs 3.0.3. Notice that Aikau, the client-side UI framework Share is built on, is included in that list as a separate component. The UI framework has been de-coupled from the rest of the platform–it now follows its own release cycle.

I consider this an awesome new “feature”, but let’s talk about real functionality actually in 5.1 Community Edition.

Content Modeling UI

If you’ve used other document management solutions and are new to Alfresco you may be surprised to realize that before this release, all content modeling had to be done by manually editing XML files. I know, I know, but the ability to define a content model in the Share UI is here now, so let’s be thankful.

Here’s how it works. If you are in the ALFRESCO_MODEL_ADMINISTRATORS group you’ll get access to a new admin panel called “Model Manager”. Creating custom types and aspects is a simple matter of point-and-click.

model-manager-scmodelBeyond just defining the model, though, the UI also allows you to define the Share form configuration.

model-manager-layout-editorIt’s pretty cool, although there are limitations:

  • Not everything supported by the underlying repository is supported by the content modeling UI. For example, I could not set mandatory aspects on a type and I could not define associations.
  • You can declare that properties must adhere to a constraint, but you can’t define the constraint once and reuse it across multiple properties.
  • I created the content model from my Custom Content Types tutorial but ran into a little problem. You can only inherit from types in an active model. I have an enterprise-wide type called “sc:doc” that my other types inherit from. The workaround is to create your enterprise-wide type first, then activate the model, then create your child types. Or you can put the enterprise-wide type in its own separate model.
  • I could not create items that inherited from sys:base or cmis:item (for content-less objects).
  • The advanced search form does not get configured to include custom types and aspects defined using the modeling UI.

The ability to define a content model without editing XML is a much-needed feature and I’m sure it will continue to evolve. It is extremely useful in its current form despite the limitations I’ve outlined above, which you can work around by using traditional techniques for defining the content model and Share form configuration.

Smart Folders

How many times have you wanted a folder to consist of query results rather than what’s physically in the file? The new Smart Folders feature gives you that capability. Unfortunately, it’s a little tedious to set up–it involves manually editing a JSON file to define your virtual folder structure. But, once you do that, it opens up a lot of possibilities.

If you aren’t sure why you’d want to define a folder as the results of a query, think about how you like to organize your files versus how other teams or departments like to organize theirs. Often, folder structures are optimized based on the work being performed. When different people with different roles work on the same content, it can become hard to create a folder structure that meets the needs of all constituents. Smart Folders allow you to set up alternative folder structures based on arbitrary criteria.

Imagine a college that facilitates internship assignments on behalf of their students. The best way to organize internship applications submitted by students depends on the person’s role. A student wants to see their own application. A counselor might want to see all of the applications for the students assigned to them. An employer wants to see the applications submitted to their company. And the internships coordinator wants to see applications by status. Prior to Smart Folders there’s not a great way to make all of these constituents happy.

In addition to creating search-defined folder structures, Smart Folders provides the ability to assign document types, aspects, and metadata values to documents automatically as they are added to the repository. For example, if you are a paralegal adding deposition transcripts to a case you no longer have to assign the cause number, client’s last name, and the fact that it’s a deposition. That gets assigned automatically based on where you uploaded the document.

To get a better feel for this, check out the Smart Folders documentation and tutorial. Alfresco has done a great job with it.

One last thing on Smart Folders–you may be wondering if a Smart Folder is accessible via CMIS. The answer is yes. Smart Folders are “normal” folder objects with an extra aspect applied (“smf:smart”). This has the potential to simplify CMIS code. Instead of putting the query in the code, you can define it in the folder template and in CMIS, just get the folder’s children which will be the query results.

Default landing page

This is another frequently-requested feature: The ability for a user to define which page should be displayed upon logging in to Alfresco Share. Now it’s easy. Just navigate to the page you want to use as the default, click your user name, then click “Use Current Page” to set.

specify-user-home-page

The next time you log in you’ll go to that page.

Become Owner UI action

Sometimes you have cases where you’d like to take over ownership of a document. One example is someone who has collaborator access. Collaborators can edit and delete documents they create because creators are owners. But maybe at some point you’d like them to be able to comment on documents in a folder but you don’t want them to edit documents even if they are the ones who uploaded them. One way to fix that is to have another user, such as a Coordinator, take over ownership of the document. This has always been possible, but before 5.1 you had to write your own script or UI action to make it happen. Now the UI action is available out-of-the-box.

AOS (Alfresco Office Services)

Alfresco Office Services is the new name for the re-implementation of the Microsoft SharePoint Protocol, which allows Microsoft Office products to enjoy native integration with the repository. If you have to use Microsoft’s office products at least you’ll be able to edit and save them directly to Alfresco.

While this is essentially the same functionality as the old SharePoint Protocol support, it does represent a significant change in Alfresco’s open source stance. Until now, all Alfresco Community Edition code has been 100% open source. Alfresco has chosen to include AOS with Community Edition, which is great, but it is distributed under a proprietary license. If that is a problem for you, the module is optional. You can still use the old open source implementation of the SharePoint Protocol, but it won’t be developed further by Alfresco. It sounds like they’ll spin it off as a separate open source project in case anyone is interested in maintaining it going forward.

jBPM has been jettisoned

Alfresco’s original embedded workflow engine was JBoss jBPM. Then, Activiti came along, and you could use either one. Eventually jBPM was marked as deprecated. With 5.1, jBPM has finally been removed from the release. Honestly, this should not be a surprise at all–you’ve had plenty of time to get your custom workflows moved over to Activiti.

SDK release lagged

I’m excited about the new release naming nomenclature and the new features. But one thing that is a little annoying is that the SDK that works with 5.1 lagged behind the release of the platform. So if you’ve got customizations that leverage the Alfresco Maven SDK, you couldn’t easily port those over in time for the release–you had to wait for the SDK. I haven’t heard whether or not this was a one-time occurrence or if this will always be the case.

Share Site type is gone

I’m not sure why it was removed or if it is coming back, but the “site type” dropdown has been removed from the “Create Site” dialog. Maybe Alfresco thought this feature wasn’t used much. If you’re using my Share Site Space Templates add-on, this will affect you because you won’t be able to specify a custom site preset that maps to your Share Site folder template. I haven’t looked at the source yet–it might not be a big deal to re-enable this.

missing-site-type-dropdownUPDATE: The Share Site type dropdown is not gone. It shows up when there is more than one type of site defined. This is a nice new feature in 5.1 because previously the dropdown would show up even if there was only a single choice.

That’s what’s new with Alfresco 5.1 Community Edition. Download it and try it out for yourself. If I’ve missed anything be sure to let me know in the comments.

And if you’d like to see any of these features live, check out my screencast:

(If the video doesn’t show up for you, here is the link).

Say goodbye to your Google Search Appliance and hello to Elasticsearch

Credit: Barabas/cc-by-sa-3.0
Photo: Barabas/cc-by-sa-3.0

Earlier this month, Google announced that it is getting out of the search appliance business. According to this article by Fortune, Google told its partners they could renew existing Google Search Appliance (GSA) customers through 2017 but no new hardware would be sold.

I have multiple clients running GSA for Enterprise Search and their experiences have been mixed. Clearly, the plug-and-play nature of a turnkey appliance was attractive. But, of course, the other side of that coin is the potential set of limitations that an appliance places on you, whether that’s in terms of cost/license, capacity, or features.

GSA customers have time to figure out their migration path. Google says they are working on a cloud-based alternative. But maybe it’s time to take a step back and consider your options.

Something big has happened since the last time you looked at Enterprise Search: It’s called Elasticsearch. The commercially-supported open source software builds on the rock solid foundation of the well-known Apache Lucene by baking in clustering and a comprehensive API out-of-the-box.

Adoption has been swift. At last year’s Elasticon conference, the company reported 20 million downloads. At this year’s conference the company announced they had hit 50 million downloads across all of their products.

Deployment options

If you want to self-support, you can set up a cluster on-prem and scale it as big as you need it for the cost of your time and some hardware. If you need commercial support you can get it from Elastic.

If a cloud-based solution is attractive to you there are several options:

  • Elastic has its own cloud offering called Elastic Cloud (formerly called “Found”).
  • QBox offers Elasticsearch hosting.
  • Amazon offers its own hosted Elasticsearch offering called Amazon ES.
  • And you can always just grab some virtual machines on your cloud provider of choice and install and run your own cluster.

The Elastic Stack provides the core search platform and a host of other tools, but it does not provide a web crawler. You’ll probably want to use Scrapy, StormCrawler, or Nutch for this, all of which are freely available as open source software.

Beyond crawlers there are a ton of different ways to get content indexed into Elasticsearch. Beats and Logstash are two Elastic products that can be used to pump data into the cluster. If you have to write your own integration, the API is fairly straightforward and is available for a number of languages as well as anything that can speak REST.

You’ll be shocked at how quickly you can stand up an Elasticsearch cluster. Where you’ll likely spend more time is on production-izing your setup and tuning for relevancy (take a look at the Relevant Search book from Manning).

Your GSA was only ever going to be good at one thing–providing keyword search for your internal documents. Elastic gives you that and so much more. You might start out using it to replace your GSA-based Enterprise Search but you’ll soon figure out that it can be used for all kinds of interesting things.