Jeff Potts' personal blog about Alfresco, content management, BPM, search, and other stuff
Category: Content Management
Enterprise Content Management (ECM), Web Content Management (WCM), Document Management (DM). Whatever you call it this category covers market happenings and lessons learned.
Independent sources within the company confirm that key leaders of the Activiti team have just resigned from Alfresco Software. Despite rumors to the contrary, this was not a coordinated effort with the team leaving en masse–each of the three departing engineers are doing so independently, for their own reasons.
Activiti is an open source Business Process Management (BPM) system that can run independently as a standalone workflow engine. Alfresco embeds Activiti in its Enterprise Content Management (ECM) platform and offers commercial support for standalone Activiti under a product offering called “Alfresco Activiti”.
Whether it is run within Alfresco or as a standalone BPM engine, Activiti is a key technology that enables businesses to streamline processes that involve both humans and other enterprise systems. The departures are awful news for the company, which has been doing a lot of marketing around automating business processes in recent months.
Alfresco created Activiti as an open source project to replace jBPM, which, at the time, did not support the BPMN specification and was unwilling to modify their license to be compatible with Alfresco’s. Alfresco hired the jBPM creator and several members of that engineering team to create Activiti as a new BPMN-compliant, Apache-licensed engine from the ground, up.
The project has been popular and successful, both in its open source community and as a commercial offering, despite hitting a bump in the road in 2013 when some of the community members decided to fork the project.
What is clear is that the Activiti open source project will continue to move forward. The most likely scenario is that one or more of the departing engineers will continue to contribute to Activiti–they’ll just earn their income from another source. I suspect that Alfresco will continue its commercial push behind BPM and will continue to leverage Activiti as it does hundreds of other open source components.
A company always suffers for a while when a good person leaves. In this case several good people are leaving. I hope Alfresco can deal with this change and move forward successfully, and I’m looking forward to seeing what these talented engineers do next.
In 2009 I wrote a blog post called “Alfresco User Interface: What are my options?” because people frequently asked for recommendations around that and they still do. Then, in 2010 at the very first Alfresco DevCon, I gave a talk called “Alfresco from an agile framework perspective“. My talk summarized how painful it was to build custom apps on Alfresco by building one simple app twice–once using Share and once using Django–and then comparing the effort it took. Django won, hands down, in terms of level-of-effort and lines of code.
Share was fairly new at the time, and it has improved a lot since 2010 in terms of ease-of-customization. In fact, a few of us met with Alfresco engineering at the conference to give feedback about the Share developer experience and Alfresco listened. The next release of Share had improved extension points. It became much easier to extend or override pieces of Share without massive copying-and-pasting of Alfresco’s code.
But there was another aspect of my talk, which boils down to this: If you are trying to appeal to developers, why are you asking us to learn a new framework? There are already frameworks out there that are extremely popular, have all sorts of tooling, and are well-documented. Use what you want to build Alfresco Share, but what can you do to make it easier for developers to build content-centric applications on Alfresco using tools we already know?
Fast-forward to BeeCon, the community-led Alfresco conference held in Brussels back in April of this year. During that conference, Alfresco announced that it was doing something about this problem: They would create libraries and components to help people create custom apps on top of the platform external to Share. Some of these will be framework agnostic. But after looking at the frameworks out there they realized they cannot deny the popularity of AngularJS, so they also want to provide customizable components that Angular developers can use.
About two weeks ago, we got our first glimpse of what this really means. Alfresco made its alfresco-ng2-components GitHub repo public followed closely by a live streamed hangout where they discussed the vision for the components (YouTube link).
Definitely clone the repo, try it out yourself, and watch that video for context. But let me try to summarize what it means for you…
What is happening to Alfresco Share, Aikau, and Surf?
Alfresco Share is the shipping web client. It is built on top of Aikau and a lower-level framework called Surf. If Alfresco is getting behind Angular, are these going away? No. Both John Newton, Alfresco’s co-founder, and John Sotiropoulos, Alfresco’s new VP of Applications, were emphatic on that point. There are no immediate plans to make any significant changes to the direction of Alfresco Share, Aikau, or Surf. Many people have customized Share–those customizations should remain safe. Plus, a lot of people use Share out-of-the-box. Alfresco still needs a web client and they are their own biggest fans when it comes to Aikau and Surf.
I am customizing Share right now. Should I stop?
No, these components aren’t going to affect you. If you have no reason to build a custom front-end on top of Alfresco, you can safely ignore this announcement. However, if you are customizing Share because you thought it would be easier than writing your own custom application, and your customizations are so significant, that it feels like you are basically re-building an application from the ground-up, with little or no re-use from what Share offers, you may want to keep an eye on this. These components will give you a jumpstart on what otherwise would have been a from-scratch effort.
What exactly is shipping right now?
Alfresco is making an initial set of components available that includes:
Login
Document list
Document viewer
Datatable
Search
Uploader
In addition to the components, Alfresco offers a Yeoman generator to help you bootstrap an Angular2 application that makes use of one or more of these components.
I am writing a custom front-end using Angular right now. Should I stop?
Alfresco is shipping Angular components that can be used to build custom apps on top of Alfresco. And you are doing the same thing. It would seem like you might want to stop and leverage the new components. Before you do that, you need to be aware of two very important caveats.
First, Alfresco is using Angular2 which has very significant differences with Angular. And Angular2 is still shipping as a release candidate. Depending on your appetite for refactoring and risk, you may or may not want to make the switch to Angular2 just yet. Still, I think it was smart for Alfresco to choose Angular2 to avoid making that move later.
The second thing is that the new Alfresco Angular2 components rely on significant enhancements to the Alfresco REST API. Those enhancements are not in any stable version of the product. They are only available starting with 201606 EA (Early Access) release, which includes 5.2.a of the Alfresco platform. There is no way to know when Alfresco 5.2 Community Edition will go to a GA (Generally Available, aka “stable”) release. Which means we also do not know when 5.2 Enterprise Edition will ship.
You should not run any Early Access release of Community Edition in production. Therefore, if you need to be in production any time soon, you will not be able to leverage these components.
This second caveat has caused a fair amount of grumbling amongst the community. There has been some talk about back-porting the new REST API to a stable Alfresco release, but, unfortunately, this cannot be done without changes to the core.
People have been developing custom front-ends on top of Alfresco using Angular and other frameworks for years without making deep changes to the core product. Alfresco didn’t have to do it in this case, but they chose to, because they wanted to clean up the API simultaneously. Basically they chose the “rip the bandage fast” approach, which I understand. However, the impact of this decision is that it will be many, many months before real world feedback on these components makes it into the shipping code. Hopefully, enough people will still be motivated enough to try them out with the EA release.
I want to try the new components, how do I get started?
First, you’ll need a running version of 5.2.a (201606-EA) with CORS enabled. I forked the gui81/alfresco Docker image and upgraded it to 5.2.a with CORS enabled and pushed the result to Docker Hub if you want to use it. Or you can download it from the wiki or Sourceforge.
The GitHub repo has a good readme, so take a look at that. I’m not going to duplicate it here, so follow those directions.
I had good luck installing the Yeoman generator and then just running “yo ng2-alfresco-app”. You’ll get a chance to specify which components you want installed into your new app–I had problems unless I selected all of them.
Yeoman generates the app. NPM is used to run it locally for development purposes. To fire it up, just run “npm start”. Once up, I was able to use the sample app to work with the back-end Alfresco repository, as shown here (click to enlarge):
It’s probably important to note that what you see running is basically a sample app or sandbox for these components. It’s up to you to write a functional app with all of the features your users need. You’ll incorporate and customize these components as part of your own development effort. You can check the documentation to learn more about how to customize each component.
Alfresco wants your feedback on these components. Don’t be afraid to report issues or submit pull requests on the GitHub repo.
Summary
I am excited to see this from Alfresco. It is too early to use these in production, but once the new REST changes ship in a GA product, you should definitely consider this as a way to jumpstart your custom apps, even if you don’t use Angular. If you’ve been looking for a way to get involved in the Alfresco community, trying these out and giving your feedback is an excellent opportunity to do so.
My friend and colleague, Peter Löfgren, recently wrote a blog post on what he sees as the two possible strategies Alfresco could pursue. He describes the two approaches as follows:
Vertical: You try to get as many as possible Community installs be converted to paying Enterprise customers. Convert from the bottom and up if you so like.
Horizontal: You try to get as many as possible to run Alfresco, even if they run the free Community version. A fair share will always want commercial support to have a professional backing. A broad approach, where you get market presence and self-sustained marketing.
Peter argues that, to date, Alfresco has used a vertical approach, which is really about pushing Enterprise Edition and begrudgingly acknowledging that Community Edition is an acceptable alternative, only when the client can’t currently justify the expense of an Enterprise Edition license.
Peter observes that the vertical approach pits Alfresco Enterprise Edition squarely against Community Edition making it very tempting for Alfresco sales people to badmouth Community Edition, because they often see it as cannibalizing their revenue. I actually don’t see this happening much anymore–sales people know that denigrating Community Edition is counter-productive because customers are savvy enough to know it is the same software and a good portion of the sales team understands the bigger picture.
Before I go any further, I should refer you to a blog post I wrote about a year ago called, “The plain truth about Alfresco’s open source ethos“. In it, I argue that Alfresco’s marketing strategy isn’t the community’s concern, and that the company is basically a “normal” software company that won’t ever be the dogmatic open source company many of us wish it was.
But Peter brought it up and I like discussing such things, so I’ll ignore my own advice and provide my take on why I think Alfresco will continue to ignore the horizontal strategy and will continue to basically act like any other traditional software company…
Here’s my take
First, let’s compare Alfresco with another commercial open source company, Elastic. Elastic is the company behind the popular search engine Elasticsearch as well as a variety of related big data and analytics tools such as Logstash, Kibana, and Beats. Like Alfresco, Elastic makes money from support, and their incentive to get people to pay for support is to offer a set of products they make available only to paying customers. Unlike Alfresco, Elastic ships a single distribution of their products. For a given release of Elasticsearch, for example, there is no difference between what a paying and non-paying customer downloads and runs. It’s just that if you want some additional value-add on top of what’s freely-available, you have to pay.
So this is an example of a horizontal pursuit as Peter describes it. The reason it is working for Elastic, though, is that their offerings are much more horizontal than Alfresco’s. Elasticsearch is more popular and more widely used in all kinds of use cases. Their download stats are impressive and increasing steadily.
Alfresco, on the other hand, is niche software. It is quite narrowly focused on document management. Yes, there are a lot of use cases within that, but it isn’t something that you see embedded in all sorts of applications like you would a database, a workflow engine, or a search engine. I suspect download stats are flat or maybe even decreasing, although this is a bit of an apples/oranges comparison as the two products are in different phases of maturity and adoption.
The other issue is one of leadership. Elastic’s CEO, Steve Schuurman, exudes open source. He was a co-founder of SpringSource, for example. Both he and Shay Bannon, the creator of Elasticsearch, have said repeatedly that Elastic will always be open source and that they’ll never have an Enterprise-only distribution of their core software. That’s the kind of leadership I expect from a commercial open source company.
Contrast that with the current leadership at Alfresco. When former CEO John Powell announced his retirement, the board could have chosen someone with open source credibility like Elastic’s Steve Schuurman. Instead, they went with Doug Dennerline. He and his lieutenants have next to zero open source credibility or experience. It is clear they were brought on solely to take the company public. For them, open source is not a driving part of their worldview. Instead, their focus is simply to build a software company and take it public, employing whatever strategy gives them and their shareholders the biggest revenue numbers year-after-year. (I don’t mean to paint this in an overly-negative light–it is what it is. I’m just trying to point out the stark contrast in motivation and philosophy between the two leadership teams).
Unfortunately, a horizontal strategy does not necessarily equate to those kind of numbers. With Red Hat as the notable exception, it is hard to find a commercial open source company with financials that Doug, the board, and investors are looking for.
Do the math. Let’s assume there are 50,000 installs of Alfresco Community Edition. I have no reason to believe that is accurate–this is just an exercise. What kind of conversion rate would you expect? You’ll probably guess too high, forgetting that Alfresco is now very expensive, even for modest installations, and that the company is still working to add more differentiation in its paid offering compared to the free product. Let’s use 2%. So that’s 1,000 paying customers, which is roughly the number John Newton disclosed publicly in a keynote several years ago. It’s probably higher now, but remember that there is attrition we haven’t accounted for and those customers have to be earned year after year.
Now, what do you think the average sales price of Alfresco is across all of their paying customers? Again, just spit-balling, let’s say it is $100k annually. Multiply that times 1,000 and that’s “only” a company with $100 million in annual revenue. If you’re looking for a $1 billion IPO, that’s not enough. (If EMC sells Documentum to someone for 10x revenue I’ll have to update this post, but I think I’m pretty safe).
In a horizontal strategy, those are your levers: Total installs, conversion rate, attrition, average sales price. For example, using the horizontal approach, to increase revenue from $100 million to $300 million you would have to triple the number of Community Edition installs from 50,000 to 150,000. Alternatively, you could keep CE installs steady at 50,000 and instead triple your conversion rate from 2% to 6%. Which seems easier?
My bet is that rather than increasing total Community Edition installs, Alfresco will find it easier to increase the conversion rate by increasing differentiation between the two products, cutting attrition by implementing “customer success programs” and consulting, and continuing to put upward pressure on the average sales price by charging more for the core product and finding new paid add-ons to sell.
The horizontal approach Peter advocates may be the one we all wish would work, but I think that ship has sailed.
This post covers the significant new features of Alfresco Community Edition 5.1. I’ve also published a YouTube video that demos the new features.
New Release Names
Alfresco Community Edition 5.1, which you may see referenced as “201602-GA”, has been released. This is the first GA release of Alfresco Community Edition using the new release naming nomenclature. Let’s talk about that first, then I’ll give you the highlights of what’s new.
In the past, Community Edition releases were assigned letters. For example, the last release in the 5.0 line was 5.0.d. Historically, Alfresco would give no clues as to whether or not they considered that a stable or final release. Subsequently, people would just grab the latest release and install it, which led to all sorts of problems. (See “Alfresco Community Edition needs sensible version labels“).
Now, thankfully, Community Edition releases are either Early Access (EA) releases or Generally Available (GA) releases. The EA releases are essentially snapshot releases that are stable enough for the community to try out and provide feedback on. The GA releases are stable builds. If you are running Community Edition in production, you should be running GA releases, not EA releases.
So the release we are talking about is “201602-GA” which means it was released in February of 2016 and it is GA. I don’t know what they are going to do if they have two GA releases in the same month, but I guess we’ll deal with that if it happens. Given that GA releases do undergo testing by Alfresco QA, which can take some time, two releases within the same month may be unlikely.
Alfresco is actually a collection of components. Community Edition 201602GA contains Alfresco Platform 5.1.e, Alfresco Share 5.1.e, Aikau 1.0.39.5, AOS 1.1, and Google Docs 3.0.3. Notice that Aikau, the client-side UI framework Share is built on, is included in that list as a separate component. The UI framework has been de-coupled from the rest of the platform–it now follows its own release cycle.
I consider this an awesome new “feature”, but let’s talk about real functionality actually in 5.1 Community Edition.
Content Modeling UI
If you’ve used other document management solutions and are new to Alfresco you may be surprised to realize that before this release, all content modeling had to be done by manually editing XML files. I know, I know, but the ability to define a content model in the Share UI is here now, so let’s be thankful.
Here’s how it works. If you are in the ALFRESCO_MODEL_ADMINISTRATORS group you’ll get access to a new admin panel called “Model Manager”. Creating custom types and aspects is a simple matter of point-and-click.
Beyond just defining the model, though, the UI also allows you to define the Share form configuration.
It’s pretty cool, although there are limitations:
Not everything supported by the underlying repository is supported by the content modeling UI. For example, I could not set mandatory aspects on a type and I could not define associations.
You can declare that properties must adhere to a constraint, but you can’t define the constraint once and reuse it across multiple properties.
I created the content model from my Custom Content Types tutorial but ran into a little problem. You can only inherit from types in an active model. I have an enterprise-wide type called “sc:doc” that my other types inherit from. The workaround is to create your enterprise-wide type first, then activate the model, then create your child types. Or you can put the enterprise-wide type in its own separate model.
I could not create items that inherited from sys:base or cmis:item (for content-less objects).
The advanced search form does not get configured to include custom types and aspects defined using the modeling UI.
The ability to define a content model without editing XML is a much-needed feature and I’m sure it will continue to evolve. It is extremely useful in its current form despite the limitations I’ve outlined above, which you can work around by using traditional techniques for defining the content model and Share form configuration.
Smart Folders
How many times have you wanted a folder to consist of query results rather than what’s physically in the file? The new Smart Folders feature gives you that capability. Unfortunately, it’s a little tedious to set up–it involves manually editing a JSON file to define your virtual folder structure. But, once you do that, it opens up a lot of possibilities.
If you aren’t sure why you’d want to define a folder as the results of a query, think about how you like to organize your files versus how other teams or departments like to organize theirs. Often, folder structures are optimized based on the work being performed. When different people with different roles work on the same content, it can become hard to create a folder structure that meets the needs of all constituents. Smart Folders allow you to set up alternative folder structures based on arbitrary criteria.
Imagine a college that facilitates internship assignments on behalf of their students. The best way to organize internship applications submitted by students depends on the person’s role. A student wants to see their own application. A counselor might want to see all of the applications for the students assigned to them. An employer wants to see the applications submitted to their company. And the internships coordinator wants to see applications by status. Prior to Smart Folders there’s not a great way to make all of these constituents happy.
In addition to creating search-defined folder structures, Smart Folders provides the ability to assign document types, aspects, and metadata values to documents automatically as they are added to the repository. For example, if you are a paralegal adding deposition transcripts to a case you no longer have to assign the cause number, client’s last name, and the fact that it’s a deposition. That gets assigned automatically based on where you uploaded the document.
To get a better feel for this, check out the Smart Folders documentation and tutorial. Alfresco has done a great job with it.
One last thing on Smart Folders–you may be wondering if a Smart Folder is accessible via CMIS. The answer is yes. Smart Folders are “normal” folder objects with an extra aspect applied (“smf:smart”). This has the potential to simplify CMIS code. Instead of putting the query in the code, you can define it in the folder template and in CMIS, just get the folder’s children which will be the query results.
Default landing page
This is another frequently-requested feature: The ability for a user to define which page should be displayed upon logging in to Alfresco Share. Now it’s easy. Just navigate to the page you want to use as the default, click your user name, then click “Use Current Page” to set.
The next time you log in you’ll go to that page.
Become Owner UI action
Sometimes you have cases where you’d like to take over ownership of a document. One example is someone who has collaborator access. Collaborators can edit and delete documents they create because creators are owners. But maybe at some point you’d like them to be able to comment on documents in a folder but you don’t want them to edit documents even if they are the ones who uploaded them. One way to fix that is to have another user, such as a Coordinator, take over ownership of the document. This has always been possible, but before 5.1 you had to write your own script or UI action to make it happen. Now the UI action is available out-of-the-box.
AOS (Alfresco Office Services)
Alfresco Office Services is the new name for the re-implementation of the Microsoft SharePoint Protocol, which allows Microsoft Office products to enjoy native integration with the repository. If you have to use Microsoft’s office products at least you’ll be able to edit and save them directly to Alfresco.
While this is essentially the same functionality as the old SharePoint Protocol support, it does represent a significant change in Alfresco’s open source stance. Until now, all Alfresco Community Edition code has been 100% open source. Alfresco has chosen to include AOS with Community Edition, which is great, but it is distributed under a proprietary license. If that is a problem for you, the module is optional. You can still use the old open source implementation of the SharePoint Protocol, but it won’t be developed further by Alfresco. It sounds like they’ll spin it off as a separate open source project in case anyone is interested in maintaining it going forward.
jBPM has been jettisoned
Alfresco’s original embedded workflow engine was JBoss jBPM. Then, Activiti came along, and you could use either one. Eventually jBPM was marked as deprecated. With 5.1, jBPM has finally been removed from the release. Honestly, this should not be a surprise at all–you’ve had plenty of time to get your custom workflows moved over to Activiti.
SDK release lagged
I’m excited about the new release naming nomenclature and the new features. But one thing that is a little annoying is that the SDK that works with 5.1 lagged behind the release of the platform. So if you’ve got customizations that leverage the Alfresco Maven SDK, you couldn’t easily port those over in time for the release–you had to wait for the SDK. I haven’t heard whether or not this was a one-time occurrence or if this will always be the case.
Share Site type is gone
I’m not sure why it was removed or if it is coming back, but the “site type” dropdown has been removed from the “Create Site” dialog. Maybe Alfresco thought this feature wasn’t used much. If you’re using my Share Site Space Templates add-on, this will affect you because you won’t be able to specify a custom site preset that maps to your Share Site folder template. I haven’t looked at the source yet–it might not be a big deal to re-enable this.
UPDATE: The Share Site type dropdown is not gone. It shows up when there is more than one type of site defined. This is a nice new feature in 5.1 because previously the dropdown would show up even if there was only a single choice.
That’s what’s new with Alfresco 5.1 Community Edition. Download it and try it out for yourself. If I’ve missed anything be sure to let me know in the comments.
And if you’d like to see any of these features live, check out my screencast:
(If the video doesn’t show up for you, here is the link).
A common request is to integrate the Alfresco Share calendar with an external calendaring system such as Outlook, Google Calendar, or Zimbra. Without an integration, people end up doing double-entry. You’ve already got a calendar that works pretty well. Why make people re-enter events in Alfresco Share?
Most people use Alfresco Share for team collaboration. The calendar doesn’t need to show everything on everyone’s calendar–that job is better left to the existing calendar server. What makes more sense is to show a few team-related events or milestones on the team’s Alfresco Share site calendar or maybe in a dashlet on the site’s dashboard.
When thinking about the problem, I realized that the calendar in Share is just another interested party in an event. Just as some calendaring systems allow you to “invite” a conference room to a meeting which effectively reserves that room for the meeting, you ought to be able to “invite” a Share site and have the Share site add that event to its calendar and update it when the event changes.
Treating the Share site as just another invitee is a non-invasive way to integrate with the calendaring system and it has the added benefit that only events in which the Share site was specifically invited will show up on the Share site calendar.
As luck would have it, the pieces to make this work already exist and they don’t require any changes to the source calendaring system. Check it out:
When you invite someone to a calendar event the calendaring system sends an iCalendar (.ICS) file as an email attachment to the invitee. The invitee’s email or calendaring client recognizes that attachment and updates the calendar accordingly.
There’s a Java library called iCal4j that knows how to parse iCalendar files. Yea for standards!
Alfresco supports receiving inbound email and you can easily bind custom logic to the creation of nodes. Alfresco creates one document for the email body and one for the ICS file attachment.
Events that show up on the Alfresco Share calendar are just content-less objects–they are instances of ia:calendarEvent.
Put those pieces together and a simple one-way calendar integration is born. The integration watches for incoming email with ICS attachments, parses the attachment, then creates, updates, or deletes the corresponding Alfresco Share site calendar object.
With this in place, all you have to do to add an event to the Alfresco Share site calendar is invite the Share site to the event from your favorite calendaring system.
But what’s the invitee name of a Share site? Great question! In Alfresco, there’s an aspect called email alias. You can add it to any folder and give it an arbitrary value. Then, when sending email to Alfresco you can specify the alias.
My integration includes code that makes sure all Share sites have a folder that can be used to store inbound email and it gives that folder an alias equal to the Share site’s short name (which is used as part of the Share URL). So if your Share site is called “test-site-1” and you normally send email to Alfresco via alfresco.someco.com, your Share site’s email address becomes test-site-1@alfresco.someco.com.
What about updates? Calendar systems have a universal identifier for every event. When calendar entries are updated or deleted, the calendaring system sends an iCalendar file just as it does for new events. Included in that file is the event’s unique ID and a flag that indicates whether the event is being created or deleted. When the integration creates the event in the Alfresco Share calendar, it stores the unique ID in the Alfresco object’s metadata which it can use later to match up subsequent update and delete requests.
How about a demo?
This video shows the integration in action. Be sure to make it full screen and select “HD”.
(If you can’t see the video, watch it on YouTube here).
What’s left to do?
This is a simple, one-way integration. It does not tell the corporate calendaring system which sites are available and it does not do a free-busy lookup. It also does not acknowledge the invitation back to the source calendaring system. I don’t consider these to be critical gaps but those features might make the integration tighter.
As a side-note, the automatic creation of an email alias for a Share site and a corresponding folder to hold inbound email (which users could then configure rules for) might be useful as a separate add-on even if you don’t need calendar integration. If you agree, let me know. Maybe the integration ought to be split into two separate AMPs.
Pull requests welcome
As usual, I welcome your participation on this project. If you find problems, fix problems, or want to make improvements, use the github project to create issues and pull requests.
Registration for BeeCon 2016 is now open. What the heck is BeeCon? BeeCon is the first-ever, independently-organized conference focused entirely on Alfresco. The BeeCon web site says it best:
Alfresco professionals and enthusiasts come to BeeCon to sharpen their technical skills and collaborate with other experts…Whether you are a developer, information professional, student, or Alfresco employee, BeeCon is the place to dive deep into Alfresco and develop the relationships which you will need to be successful in the coming year.
The conference is organized by the Order of the Bee, an independent community focused on Alfresco.
Who Will Attend?
BeeCon is an event organized by and targeted towards the Alfresco community. It is built around the idea that what makes our community great is its open, collaborative spirit. And that, from time-to-time, it is important to meet face-to-face to learn from each other, hash out ideas, strengthen personal relationships, and just have fun.
If Alfresco is just a piece of software to you, then this is a conference with a lot of technical how-to’s that will help you get your project done, and you should come for that reason. When you arrive, though, you’re going to find out that a lot of people have crossed oceans and continents to be in Brussels because not only is the software important, but because, as a community, we have a lot of work to do. And the people who care about the Alfresco community are using this event to get organized and to map the way forward.
If you love sales pitches and marketing fluff you should sit this one out. But if you…
want to learn more about the technical details from experts;
are already running Alfresco in your organization, whether that’s Enterprise or Community Edition; or
want to help shape the future of the community and the platform
…then you need to attend BeeCon 2016.
More than a Meetup
This is more than a meetup. It’s a real two-day conference with keynotes, tracks, and a hack-a-thon. The goal is to make it similar to past events like DevCon with really great content and outstanding people, but without the big budget (or price tag).
You can register now for about 60 Euros. If you wait the price goes up to about 90 Euros.
Support from Alfresco and Other Sponsors
The BeeCon team has focused on keeping things practical and inexpensive. But events like this simply cannot succeed without help from sponsors. This year, CIRB-CIBG is providing the venue, A/V equipment, and WiFi, which is amazing because those three items are the biggest in terms of cost for any event. What’s even more amazing is that we enjoy additional support from a number of sponsors including Alfresco, Contezza, ITD Systems, keensoft, VDEL, and Xenit. You should thank these folks when you see them.
Stay Tuned for the Detailed Agenda
The program team received a number of speaking submissions from Alfresco engineers and community members from all over the world. They are busy reviewing those and will get the conference web site updated as things solidify. The team is picky–they want sessions to be high quality and packed with information you can use on your Alfresco projects right away. I’m looking forward to seeing the finished agenda, but I’m not going to wait to register.
Space is Limited, Do Not Wait to Register!
While you’re thinking about it, complete your registration. It’s only 60 Euros. I’ll bet you can slip that into an expense report without much fuss. And when you bring the things you learn back to the office, you’ll win respect and adoration from your boss and coworkers. Not bad for 60 Euros.
When making your travel plans for Brussels, remember that we’ll be getting together Wednesday night, April 27, for a welcome reception. The conference runs two days, April 28-29. Then, whomever is interested can come with us to the medieval city of Bruges on Saturday, April 30, for a day of sightseeing. I’ve been to Bruges–it’s gorgeous. You won’t want to miss it. Plus, it will be nice to hang out with your favorite community members, Belgian-style.
I look forward to seeing you in Brussels in April!
One of my clients came to me with a problem: Despite being a much-admired Fortune 500 company that leads its competitors in the travel industry in customer satisfaction and profitability, their web site, through which the vast majority of their revenues flow, was still mostly static. That by itself is not a huge problem, but they felt like they weren’t able to target content based on their customers’ needs and interests as well as they could with a more dynamic content engine.
It just so happened they were about to re-implement their site from mostly server-side to mostly client-side which is a huge undertaking. They figured that would be a pretty good time to add a dynamic content service to the mix, so they called me.
From Static to Dynamic
The diagram below depicts the high-level setup before the introduction of the content service.
This is pretty standard for sites like this. The Marketing Team edits content in a Content Management System (CMS), which in this case is Interwoven. Through various processes, binary files (mostly images), system data (things like lists of destinations and hotels), and content fragments are published out of Interwoven to destinations accessible by the e-commerce application.
A content fragment is literally a piece of content. It might be a promotion of some sort. Or it could be some text that gets used as part of a banner. The challenge using this setup is that content fragments are static files that live on the file system. If you want to show a different fragment based on something you know about the user you have to generate every permutation you might want ahead-of-time, publish them all, then use logic in the application to decide which one to use.
One obvious way to address this is to publish content fragments in a relational database and then code the front-end app to query for the right content. That wasn’t appropriate here for a few reasons:
The front-end is being migrated to a collection of Single Page Applications (SPA’s) written in JavaScript. It’s easier for those pages to call a RESTful API to get JSON back. Yes, you could still do that with a relational database and a service tier, but the client was looking for something a little more JSON-native.
The structure of the content changes over time. We wanted to be able to accept any kind of content fragment the Marketing Team or SPA developers could think of and not have to worry about migrating database schemas.
The anticipated style of queries needed to find appropriate content fragments was more like what you’d expect from a search engine and less like what you might put in a SQL query–we needed to be able to say, “Here is some context, now return the most appropriate set of content fragments for the situation,” and be able to use relevancy scoring to help determine what comes back.
So relational databases were ruled out in favor of document-oriented NoSQL repositories. Ultimately, Elasticsearch was selected because of its ease of clustering, high performance, unified REST API, availability of commercial support, and add-ons such as Shield, Marvel, and Watcher that make it easier to integrate with the rest of the enterprise.
Introduction of a Content Delivery Service
The first thing we did was stand up an Elasticsearch cluster, load some test data, and beat the heck out of it (see “Using JMeter to Test Elasticsearch“). Once we were satisfied it would be able to handle more than the expected load we moved on to the service.
The Content Delivery Service sits between Elasticsearch and the front-end applications. Its purpose is to abstract away Elasticsearch specifics and to protect the cluster by providing a simple, read-only REST API. It also enforces some light business logic such as making sure that only content that is currently effective according to its publication and expiration date is returned.
The diagram below shows the content infrastructure augmented with Elasticsearch and the content delivery service.
As seen in the diagram, Interwoven is still the source of record and the primary way Marketing manages their content. But now, content fragments and system data are published to Elasticsearch. The front-end Single Page Apps ask the Content Delivery Service for content based on some set of context. The content is returned as a collection of JSON objects. The SPAs then take those objects and format them as needed.
Content Objects are Pure Content
A key concept worth emphasizing is that a content object is pure content. It contains no markup. It might have some properties that describe how it is expected to be used, but it is completely lacking in implementation. This has several benefits:
Content objects returned by the Content Delivery Service can be used across any and all channels (such as mobile) rather than being specific to a single channel (such as web).
Within a given channel the same object can have many different presentations.
Responsibilities are cleanly separated: The content service provides content. The front-end applications style and present the content for consumption.
This was a bit of a departure from how things used to be done. In the bad old days presentation was always getting mixed up with content which severely limits reuse.
Micro-services Provide Administrative Features
I mentioned earlier that the Content Delivery Service is read-only. And in my previous diagram I showed Interwoven talking directly to the Elasticsearch cluster. In reality, we don’t let anyone talk directly to the Elasticsearch cluster. Instead, all writes have to go through the Content Management Service. This ensures that we know exactly what is going into the cluster and who is putting it there.
The other role the Content Management Service plays is JSON validation. When new types of content objects are developed we use JSON Schema to codify the structure. When a person or system posts a content object to the Content Management Service, the service validates the object against its JSON Schema before storing it in Elasticsearch.
In addition to the Content Management Service we also implemented a Scheduled Job Service. As the name suggests, it is used to perform administrative tasks on a schedule. For instance, maybe content needs to be reindexed from one cluster to another in a lower environment. Or maybe content needs to be fetched from a third-party and written to the cluster. The Job Service is able to talk to either the Content Management Service or Elasticsearch directly, depending on the task it needs to execute.
All of the administrative services are independently deployed web applications that sit behind an API Gateway. The Gateway leverages the Netflix Zuul Proxy. It is responsible for authenticating against LDAP and creating a shared session in redis. It gives the content admin team a single URL to hit and isolates authentication logic in a single place.
The diagram below shows the fully-realized picture.
A few key components aren’t on the diagram. We use Shield to protect the Elasticsearch cluster. Shield also makes it easy to configure SSL for node-to-node communication and provides out-of-the-box LDAP integration. With Shield we can map LDAP groups to roles and then grant roles various privileges on our Elasticsearch cluster and its indices.
We use Watcher to monitor cluster health and job failures that may happen in the Scheduled Job Service. The client has their own enterprise alerting and monitoring solution, but Watcher gives the content management team a flexible, powerful tool for keeping track of things at a level that is probably more granular than what the enterprise ops team cares about.
Ready for the Future
With Elasticsearch and a few relatively small services on top of that, this travel giant now has what it needs to provide its customers with a more customized online experience. Content can be targeted to the users it is most appropriate for using any kind of context the Marketing team can come up with. As the front-end commerce app evolves, new types of content objects can be added easily and be served to the front-end with no schema or service changes required. And it’s all built on commercially-supported open source software.
Five years ago I wrote a blog post called, “Alfresco, NoSQL, and the Future of ECM“. Today is “Back to the Future Day”–the exact date Marty McFly time-traveled to in the movie Back to the Future. The movie made many observations about what life would be like on October 21, 2015–some were spot on, some not so much. I thought it fitting that we take a look at my old blog post and see where we are now with regard to content repositories and NoSQL.
One point of the post was that NOSQL might be a more fitting back-end for content repositories than relational:
But why shouldn’t the Content Management tier benefit from the scalability and replication capabilities of a NOSQL repository? And why can’t a NOSQL repository have an end-user focused user interface with integrated workflow, a form service, and other traditional DM/CMS/WCM functionality? It should, it can and they will.
This has definitely turned out to be the case. New content management solution vendors like CloudCMS have built their platform on NOSQL technology while older vendors, such as Nuxeo, have started to integrate NOSQL into their solutions.
Open source projects are also taking advantage of the technology. Apache Jackrabbit provides an implementation of the JCR standard. Its “next generation” offering, Jackrabbit Oak, is essentially JCR with MongoDB as the back-end.The second point of the post was that as NOSQL repositories become more widely adopted, they compete directly with content repositories in use cases where those content repositories are used primarily as a back-end for developers’ custom content-centric solutions.
In other words, 5 or 10 years ago, if you were a developer looking to implement a custom application, and you wanted something other than a relational back-end, you might build your application on top of something like Alfresco. Now developers may be less likely to go that route. That’s because today there is an explosion of stacks out there. Many of them assume a NOSQL back-end. Look at mean.io as just one example, which combines Node.js, Express (the Node.js web framework), AngularJS, and MongoDB and wraps it up with time-saving tooling.
Many people use Alfresco as a back-end. Their front-end uses a RESTful API implemented as web scripts to talk to the repository. The value the repository brings to the table is the ability to store documents in a hierarchy along with custom metadata defined in a content model. They may not be using Alfresco Share or much of the other functionality that Alfresco bundles with their offering–for their custom solution Alfresco is just a repository. When it is used like this, Alfresco is doing nothing more than what NOSQL repositories offer, and, in fact, it does less because NOSQL repositories have a more flexible schema and are built to be clustered and massively distributable–for free.
Years ago, Alfresco shifted its focus away from developers looking to build custom solutions on top of a bare repository. Its developer outreach is now more about customizing Alfresco Share and the underlying repository. Nuxeo, on the other hand, has doubled-down on its developer focus. I’ll spend some more time on this in a future post.
I guess this trend wasn’t terribly hard to predict five years ago, but it does feel kinda nice to see it come to pass. Now, if I could just have a hoverboard.
Earlier this week, in a post to a public mailing list, Ole Hejlskov, Developer Evangelist at Alfresco, announced that the company will not be putting on its annual conference, Alfresco Summit, this year as originally planned. Instead, the company is focusing on smaller, shorter, sales-oriented events which have been very successful in several cities around the globe.
Ole said that Alfresco will be adding developer content to its Alfresco Day events, which have historically been mostly end-user and decision-maker focused. In contrast, Alfresco’s yearly events started out as developer-focused conferences, but in recent years had a more balanced agenda with both technical and non-technical tracks.
Alfresco had announced earlier in the year that their annual conference would be in New Orleans in November. In each of the last five years the company put on two conferences–one in Europe and the other in United States. For 2015 the plan was to have a single conference only in the U.S. which drew criticism from the community that skews heavily toward a non-U.S. demographic.
When the community realized Alfresco Summit 2015 would be held only in the U.S., an independent community organization called The Order of the Bee began making plans to hold their own conference in Europe. Alfresco says it will support the community’s efforts to hold its own event and wants to explore “…ways in which participation from Alfresco corporate makes sense”.
I understand where Alfresco is coming from. Annual conferences are expensive in both real dollars and the time and attention it takes to plan and execute. When you multiply that times two it obviously represents an even bigger investment.
You also have to look at what Alfresco gets out of the conference. Alfresco is increasingly sales-focused. The conference has historically been focused on knowledge-sharing and camaraderie. Yes, there were deals closed at Alfresco Summit but it was not geared towards selling. It was more about coming together to share stories, good and bad.
The Alfresco Day events are unabashedly sales and marketing. The attendees (and they get very large turnouts) know this which means Alfresco does not have to apologize for coming off too sales-y. Multiple cities with hundreds of prospects is a better investment for them than two cities with 1400 attendees who are existing customers and community members.
As the guy who led DevCon and Alfresco Summit and together with my team grew it year after year, it is weird to see Alfresco cancel the conference for 2015. I was looking forward to attending.
As a member of The Order of the Bee, I’m intrigued by the challenge of using an all-volunteer organization to potentially put together a replacement conference of some sort. If you have any interest in helping and you did not see my email to the mailing list, we’ll probably be meeting next week to get organized. Reach out to me and I’ll add you to the invitation.
There was a small flare-up on the Order of the Bee list this week. It started when someone suggested that the Community Edition (CE) versus Enterprise Edition comparison page on alfresco.com put CE in a negative light. In full disclosure, I collaborated with Marketing on that page when I worked for Alfresco. My goal at the time was to make sure that the comparison was fair and that it didn’t disparage Community Edition. I think it still passes that test and is similar to the comparison pages of other commercial open source companies.
My response to the original post to the list was that people shouldn’t bother trying to get the page changed. Why? Because how Alfresco Software, Inc. chooses to market their software is out-of-scope for the community. As long as the commercial company behind Alfresco doesn’t say anything untrue about Community Edition, the community shouldn’t care.
The fact that there is a commercial company behind Alfresco, that they are in the business of selling Enterprise support subscriptions, and at the same time have a vested interest in promoting the use of Community Edition to certain market segments is something you have to get your head around.
Actually there are a handful of things that you really need to understand and accept so you can be a happy member of the community. Here they are:
1. CE is distributed under LGPLv3 so it is open source.
If you need to put a label on it and you are a binary type of person, this is at the top of the list. Alfresco is “open source” because it is distributed under an OSI-approved license. A more fine-grained description is that it is “open core” because the same software is distributed under two different licenses, with the enterprise version being based on the free version and including features not available in the free version.
2. Committers will only ever be employees.
There have been various efforts over the years to get the community more involved in making direct code contributions. The most recent is that Aikau is on github and accepting pull requests. Maybe some day the core repository will be donated to Apache or some other foundation. Until then, if you want to commit directly to core, send a resume to Alfresco Software, Inc. I know they are hiring talented engineers.
3. Alfresco Software, Inc. is a commercial, for-profit business.
Already mentioned, but worth repeating: The company behind the software earns revenue from support subscriptions, and, increasingly, value-added features not available in the open source distribution. The company is going to do everything it can to maximize revenue. The community needs this to be the case because a portion of those resources support the community product. The company needs the community, so it won’t do anything to aggressively undermine adoption of the free product. You have to believe this to be true. A certain amount of trust is required for a symbiotic relationship to work.
4. “Open source” is not a guiding principle for the company.
Individuals within the company are ardent open source advocates and passionate and valued community members, but the organization as a whole does not use “open source” as a fundamental guiding principle. This should not be surprising when you consider that:
“Drive Open Innovation” not “Open Source” is a core value to the company as publicly expressed on the Our Values page.
The leadership team has no open source experience (except John Newton and PHH whose open source experience is Alfresco and Activiti).
The community team doesn’t exist any more–the company has shifted to a “developer engagement” strategy rather than having a dedicated community leadership or advocacy team.
Accept the fact that this is a software company like any other, that distributes some of its software under an open source license and employs many talented people who spend a lot of their time (on- and off-hours) to further the efforts of the community. It is not a “everything-we-do-we-do-because-open-source” kind of company. It just isn’t.
5. Alfresco originally released under an open source license primarily as a go-to-market strategy
In the early days, open source was attractive to the company not because it wanted help building the software, but because the license undermined the position of proprietary vendors and because they hoped to gain market share quickly by leveraging the viral nature of freely-distributable software. Being open was an attractive (and highly marketable) contrast to the extremely closed and proprietary nature of legacy ECM vendors such as EMC and Microsoft.
I think John and Paul also hoped that the open and transparent nature of open source would lend itself to developer adoption, third-party integrations and add-ons, and a partner ecosystem, which it did.
I think it is this last one–the mismatch between the original motivations to release as open source and what we as a community expect from an open source project–that causes angst. The “open source” moniker attracts people who wish the project was more like an organic open source project than it can or ever will be.
For me, personally, I accepted these as givens a long time ago–none of them bother me any more. I am taking this gift that we’ve been given–a highly-functional, freely-distributable ECM platform–and I’m using it to help people. I’m no longer interested in holding the company to a dogmatic standard they never intended to be held to.
So be cool and do your thing
The “commercial” part of “commercial open source” creates a tension that is felt both internally and externally. Internal tension happens when decisions have to be made for the benefit of one side at the expense of the other. External tension happens when the community feels like the company isn’t always acting in their best interest and lacks the context or visibility needed to believe otherwise.
This tension is a natural by-product of the commercial open source model. It will always be there. Let’s acknowledge it, but I see no reason to antagonize it.
If you want to help the community around Alfresco, participate. Build something. Install the software and help others get it up and running. Join the Order of the Bee. If you want to help Alfresco with its marketing, send them your resume.
Independent sources within the company confirm that key leaders of the Activiti team have just resigned from Alfresco Software. Despite rumors to the contrary, this was not a coordinated effort with the team leaving en masse–each of the three departing engineers are doing so independently, for their own reasons.
In 2009 I wrote a blog post called “
Five years ago I wrote a blog post called, “
Earlier this week, in a