Consolidate THEN Integrate

July 14, 2008

SOA is seen by many as a strategic approach to solve the application integration problem.

Organisations need to integrate for one of two reasons. They either want to share data or they want to synchronise data between disparate systems.

Sharing data is a worthwhile pursuit. For example, when we talk about things like ‘single customer view’ we are talking about collating everything we know about a customer from all the systems we have. Having an enterprise-wide view of critical information, rather than an application-centric view, allows for better decision making.

Synchronising data on the other hand is generally a waste of time. The need to synchronise come about because an enterprise has many different systems to do the same job. For example, the sales department might have one CRM system to manage prospects and the customer service department has a different CRM system to manage customers. Now it doesn’t take a Rhodes Scholar to figure out that there will be some overlap between these two systems. So what’s the solution here? The truly ‘strategic’ decision would be to consolidate the two systems into one. Integrating the two systems is a tactical solution. It’s important to remember that just because a plan of action is long-term doesn’t necessarily make it strategic.

As part of any SOA strategy, you should look to consolidate systems first and then integrate whatever is left.

Footnote: SOA folk will know that in order to enable ‘sharing’ you will need some degree of ‘synchronisation’, but in this case synchronisation is an enabler – a means to a worthwhile end – rather than an end in itself.


From Supply Chain to Business Ecosystem via SOA

June 8, 2008

Most of the discussions surrounding SOA focus on the individual organisation. It appears that people are looking to service-orient their own business before looking outside. There’s certainly nothing wrong with this. However, it seems that focussing internally is the ‘default’ position, rather than a considered decision.

Supply chains are complex things and they are the lifeblood of any business. While there’s always room for improvement most organisations have a pretty good handle on them (if they didn’t they’d be out of business).

A traditional supply chain looks like this (simplified):

simple supply chain

But things are changing.

Technology is improving at a rapid rate and we’re living in an increasingly connected world. Consequently, businesses will come under increasing pressure to connect to the outside world. Businesses will be forced to move from the traditional (linear) supply chain to a business ecosystem:

business ecosystem

A linear supply chain is relatively easy to manage. You have two touch points with the outside world. The procurement/logistics functions deal with suppliers and the sales/accounts receivable functions deal with customers. Businesses have been doing this for a long time.

Business ecosystems, on the other hand, have a complex array of relationships and each relationship has to be managed. This can’t be done without the help of technology. Notice I didn’t say it can’t be done efficiently. I said it can’t be done at all.

Service oriented architectures are ideally suited to providing the technical foundation for a business ecosystem. The ROI of outward looking projects may far exceed the ROI of any internal project. There’s no rule that says you must ALWAYS start internally.

Something to think about.

Mashups Are About Assembly. SOA is about Integration

June 6, 2008

Anyone that thinks that eye-candy mashups created by users ‘at the glass’ is the beginning of the end for IT Departments is mistaken. Realistically, such mashups will only allow users to assemble – not integrate. This is a subtle but important distinction.

A good example that illustrates the difference between assembly and integration is the good old Lego block.

The size and shape of Lego blocks didn’t happen by accident. They look the way they do because someone made a conscious decision that the chosen design would maximise flexibility. The company knew that they could use the same base set of blocks and combine them into an endless array of products. What’s more, buyers could build things that the Lego company hadn’t even thought of.

What’s this got to do with mashups and SOA?

When we build an SOA we build a range of services that all conform to a particular design or standard (i.e. shape of the Lego block). This is easier said than done. Building these services takes a lot of thought and a lot of work. Integration is hard.

When we’ve designed and built our services (blocks), we don’t stop there – we then combine (or package) those services into a useable product (i.e. Lego block set) that someone will actually use – for example composite applications or portals.

And here’s where it gets interesting. Users can use the product we’ve produced or they can take our base services and re-assemble them (mash them up) in any way they choose. Probably in ways that we hadn’t even thought of.

So, IT has done the hard integration work and designed the blocks in a way that will make them flexible and re-useable. The business can then take those building blocks and do whatever they like with them. Everyone is happy.

Consider the alternative scenario where users bypass IT and go their own way. The user will be faced with a toy box full of things that just don’t fit together: Lego blocks, Meccano, Steel Tec and Minibrix. Before the user can assemble anything useful they have to figure out how to integrate everything by devising all sorts of connectors and converters. Even if mashup platforms provide sophisticated tools to create these connectors and converters, let’s not forget that IT already has industrial strength integration tools and the skills/experience to use them to their full potential – but integration is still hard work. There is a limit to the functionality that can be reasonably used by end-users. This scenario is clearly a sub-optimal approach.

The fact is – business and IT must work together. Each side brings unique skills to the table. You can’t have a situation where end-users try to bypass IT and in turn, IT waits for end-users to fail.

No More Groups, Committees, Centres or Offices

June 5, 2008

If you’re not familiar with Todd Biske’s blog then do yourself a favour and have a read. Todd is pragmatic and business focussed. And he’s also an Enterprise Architect. A rare combination. I normally agree with Todd but his last two posts here and here got me thinking. These posts talk about Centres of Excellence and whether they should be temporary or permanent. My question is: why do we need them at all?

Todd quotes a meeting of the SOA Consortium which talked about the skills required to operate a SOA Centre of Excellence. They are: project/portfolio management, service design, business knowledge, technical aptitude, communication, teaching and governance. Todd also mentions Gartner’s idea of an ‘Integration Competency Centre’.

As far as I’m concerned ‘SOA’ and ‘integration’ are subsets of enterprise architecture. Furthermore, all of the skills listed above should be found in your existing Enterprise Architecture Group. So why do we need more centres, groups, committees or offices? Todd’s answer:

While enterprise architecture has the appropriate scope of visibility and influence on the technical side, they’re not the best group for handling organizational decisions.

This is a damning indictment of the EA profession. Worse than that: it’s true.

That said, I think there are three main reasons why creating a new group might be a bad idea.

1. Staffing
How is this new team going to be staffed? I’ll assume for the moment that getting business people will be easy (big assumption). But what about the technical people? Your best guys will probably be in the existing EA Group. In most organisations SOA is not the only game in town. Who is going to look after the other things when the best architects are poached by another group? If the EA Group is large it will typically have lots of specialists – so who will you take and who will you leave behind (take the information architect and leave the infrastructure architect)? And how will you fill the resulting hole? If the EA Group is small it will usually consist of generalists but taking even one person from a small group will leave a significant hole in its ability to deliver.

2. Overlapping Responsibilities.
There will be huge overlap between the functions of the new group and the existing EA Group. When responsibility is shared, accountability suffers. No ONE is accountable for anything. How will you divide the responsibilities between the two groups? How ill you manage the inevitable overlap (however small)? Will the business have to go over everything twice?

3. Cross-functional teams are usually temporary.
Cross functional teams are nothing new. But typically these groups are short-term and temporary – formed for a specific purpose (project) and then dissolved. For example, businesses may use these teams to implement new quality systems, oversee a merger or implement a new CRM system. The simple reason cross-functional groups are temporary is because the best people are reluctant to get off their career path within the business and move into a ‘special project’ permanently. SOA is not a ‘project’ and it’s certainly not ‘short term’.

A Better Solution?
If your EA Group can’t deliver on everything that a Centre of Excellence would be expected to – then fix your EA Group! Typically the two main problems are a lack of credibility and a lack of business focus. Both of these can be solved by making the EA Group more business focussed. This can be done in two ways:

Short term solution. Physically bring business people into the EA Group. You’ll then have a cross-functional centre of excellence without actually creating one from scratch. Of course, there won’t be too many business people volunteering for the job, but this is not an insurmountable problem. A properly functioning EA group has enormous potential for delivering business value – and the business person responsible for ‘finally getting those techies to see things from a business perspective’ will be a popular fellow at the executive golf day. So there’s plenty of upside for any business person wishing to take on the challenge. And it should only be a temporary assignment.

Long term solution. To do their jobs properly EA’s MUST build relationships with all parts of the business. If the business avoids engaging with the EA team (or vice versa) then there’s a major problem. If this isn’t fixed then you might as well dismantle the EA group as it would be impossible for them to be adding value to the business.

In my opinion, creating a new group to overcome the shortcomings of an existing one is akin to saying “my house is dirty so I’ll go buy a clean one” (thanks Krusty of Simpsons fame for this quote). The new house will get dirty eventually. Similarly any new group created will eventually face and same challenges that the EA Group did (the day to day ‘stuff’), and probably end up in the same position.

Mashups and SOA: The Last Mile

May 26, 2008

I love great analogies – they allow us to explain complex things in simple terms. I found a real gem today. Unfortunately I can’t find the blog post in which I read it. In the post, mashups were described as SOAs “last mile” (I’m paraphrasing here because I can’t remember the exact quote). I don’t think I’ve read a more succinct description of the relationship between SOA and mashups.

The “last mile” is a telecommunications term used to describe the section of a telecommunications line that runs from your nearest exchange to your home. I think this is a great analogy for mashups and SOA. Here’s why.

The telecommunications system is a complex thing – exchanges, switches, cables, satellites and who knows what else. Thankfully, all of this complexity is hidden from the end user. We use the telephone network every day through a simple ‘user interface’ – the telephone. Most of us don’t know (or even care) how it all works. For the end user, the true value of the telecommunications network is realised through the ‘last mile’ of wire that connects the telephone to the telecommunications network.

SOAs are similarly complex – just like the telecommunications network, there’s plenty going on under the hood but if implemented properly, all of the SOA complexity is hidden from the users. And mashups may be the tangible thing that allows users to realise the full value of the infrastructure and plumbing that is SOA.

Without the telecommunications network, the telephone is a useless gadget. Conversely, without the telephone that sits on your desk, the telecommunications network is a nebulous concept without a practical use.

I believe that mashups need a proper SOA foundation to realise their full potential. I also believe that building a SOA that doesn’t offer tangible benefits that the business can see and touch is a pointless exercise.

PS: If anyone knows the post I’m talking about please comment below or email me.

Black-Box and White-Box SOA Services

May 21, 2008

Recently I had an interesting conversation with a business process professional, on the subject of SOA services. I started talking about ‘white-box’ and ‘black-box’ services, assuming they were clearly understood terms. It turns out that they weren’t. Here’s how I explained the two terms, and why they’re important.

The services in your SOA need to be ‘black-box’. If ‘white-box’ services creep into your architecture sooner or later you’ll have problems because white-box services increase coupling, thus violating one of the fundamental tenets of SOA (that services should be loosely coupled).

So what’s the difference between white-box and black-box?

Everyone is familiar with airline travel so let’s use that as an example.

Let’s say you want to go on a trip to attend a sporting event.

Now there are two ways of achieving your goal of attending the sporting event.

The Black-Box Way
You find a good travel agent and you tell them “I want to go to ‘such and such’ a sporting event on this date and I want you to arrange everything I need to get me there”.

They’ll ask you to fill out a form with all the information they need to fulfil your request: dates/times, credit card details, name etc, In SOA terms this form is the ‘contract’ through which you interact with the travel agent ‘service’.

You don’t care what they do with that form, who they have to speak to or how they go about arranging everything. It could be all online in real-time or maybe the pick up the phone and make all the bookings manually with each party concerned (event ticket office, airline, airport transfer company etc). You tell them WHAT you want and leave them to it. It’s “black-box”.

The White-Box Way
You find a not-so-good travel agent. They ask you to fill out 5 forms:

  • A form for booking a taxi from your house to airport through Tristate Taxis Co.
  • A form for booking an airline ticket through Qantas.
  • A form for booking a transfer from airport to hotel through Fred’s Airport Transfers
  • A form for booking a hotel room at the Hyatt.
  • A form for booking a event tickets through Ticketek

In this situation you can see inside the travel agent ‘service’. You know exactly HOW they’ll go about getting you to your sporting event. You know step-by-step what is involved and who is involved.

Why is this bad? Because the travel agent ‘service’ wont be able to change anything about its internal implementation without affecting consumers:

  • What if you want to use a different airline?
  • What you need a transfer from the airport straight to the event, but your luggage needs to be sent to the hotel?

Keep your SOA services black-box.
That is a simplistic example. The core concept is: consumers of services should only be concerned with WHAT your service does. Services should not leak any details of HOW they do


May 19, 2008

A few days ago I discussed how Web oriented Architecture (WOA) relates to SOA. Today I’ll touch on EDA. Some aren’t sure whether to go with EDA or SOA. The great news is, that’s not a decision you need to make. It’s like asking yourself a nonsensical question like “should I have a drink or should I quench my thirst?”

Let me cut to the chase: EDA is “event driven SOA”, it’s as simple as that.

If you design your SOA around numerous independent, autonomous services which are unaware of each other and merely publish events which other services subscribe to, then you have event-driven SOA. Or EDA. Take your pick.

So if the two terms (SOA and EDA) are so intertwined where has the confusion come from?

The answer is: EDA is a subset of SOA. It is quite possible that your SOA may use RPC style ‘command messages’. This is a technique that falls outside the scope of EDA – but it is quite legitimate in SOA. When I ‘legitimate’ I mean ‘if used correctly, with implications clearly understood’. It should be the exception rather than the rule.

So. If you have a drink then you’ll quench your thirst. If you create an event-driven SOA then you’ll have EDA. Forget about acronyms – they’re used by vendors and analysts trying to create markets or carve out niches for themselves. Architects should be looking for the substance behind the hype.