Technology Thoughts

Tuesday, October 31, 2006

The legacy ESB

3-5 years in the future from now on, many companies who thought that they bought a SOA by buying an ESB will be faced with the issue of buying some new infrastructure, maybe a new ESB, packed with new features. But they won't want to replace their old one, full of cranky old orchestrations who nobody in the corporation does any more how were done and that nobody want to touch because they just work...

Then the company will realize that they will have a legacy ESB. And if they buy something new, it will run just as fine, being integrated in the new infrastructure. Hopefully they will learn, then, that the ESB is not the center of a SOA at all, but just another provider of services, just as so many other things in a SOA.

The typical three-tier SOA (legacy-orchestration-frontend) is not a reference Service-Oriented Architecture; it is just a way of building integration services / applications.

Friday, October 06, 2006

SOA and reuse

There seems to be a debate about the true extent of reuse in SOAs. E.g. David Chappel, Joe McKendrick, or Theo Beack have blogged about it.

I think that inside an organization reuse is perfectly possible, because in many cases there is only a single context. E.g. inside of a company there should be room for only one logic for a Purchase_Order_service.

The problem for reuse is across a large market of organizations, or for larger organizations with different context. E.g. to be able to buy a "generic" Purchase_Order_service and adapt it for your context.

Theo Beack says that "the context should be elevated to the process layer and NOT encapsulated within the service". But I think this would result in services doing tasks of very limited usefulness, since the actually valuable business logic would be inside business processes, which would create the context by linking services. One can envision these business processes being traded across companies (the context changing thanks to using different services), but I think there is not yet a framework for this. But one for services is emerging, so it may be better to use it.

I think that the key to reuse may be to factor out funcionality as much as possible: create many small services doing concrete things, and relying themselves on other small services. Then all these small services can have a particular implementation inside a particular context. E.g. Purchase_Order_service would rely on Product, Customer, Invoicing etc services which would be specific for the context of an organization. Interface-oriented design is key to this.

This would make services to depend on other services, resulting in more thigt-coupled services, but in exchange they would deliver much more business value than isolated services.

Wednesday, October 04, 2006

About the SCA and SDO

During some months now, many software vendors lead by IBM have been involved in the SCA initiative (well it is actually Open SOA, whose two main pillars are SCA - Service Component Architecture and SDO - Service Data Objects).

As far as I understand it, the intention of SCA is to create a standard environment (APIs, models, containers) in which services of a SOA can run and interoperate, no matter which implementation or platform they have, or even which type of components they use, since it aims to integrate Java objects, EJBs, C++, or whatever. This is, it is sort of a standard, vendor-neutral middleware for creating services; the specification of a SOA platform which every vendor can then implement.

For its part, SDO aims to create a standard way for service implementations to access any kind of data, from RDMBS to XDBMS through web services.

So all in all to me it seems much like J2EE: allow both to create distributed components and access data. Of course, not limited to Java or even to a single networking protocol, but for every platform.

When I first saw the creation of the initiative I thought that this was just a rebranding of proprietary IBM technologies (things called SCA and SDO had been available at IBM for some time before that). Well it seems something like that, only that hopefully it will be truly open. Although I can guess that IBM, being the inventor, will have much advantage in its implementations on the rest of the consortium. But who knows.

At any rate, I still do not see the point of SCA. For me, the platform that allows for interoperability of every kind of software component in any implementation or platform is web services. SCA should allow to have interoperable services other than web services, but I think that WS-* is enough and that there is no need of any more.

But we will see, since the success of these initiatives rely on the end upon who supports them (e.g. it does not matter you like CORBA more than SOAP, since SOAP is supported by every vendor), and Open SOA has an impressive list of them (BEA, Oracle, TIBCO, Sun and several others). However, they happen to be only Java vendors: Microsoft is not there. And if SCA will not properly supported in Microsoft platforms, it means that it will not be interoperable enough, as WS-* is.

Tuesday, October 03, 2006

The ESB and the SOA

(I am replacing this post, because the original one was too obscure and not even me really understood it after some time of having written it. So I hope the new version transmits better which I think has to be the role of an ESB in a SOA.)

ESBs are, for one thing, a fuzzy category of middleware products with a large variety of capabilities. From these capabilities, I think there are two that ESBs are well suited to perform:

  • Managed communications, including support for different protocols, asynchronous and publish/subscribe communications, and reliable communications (in general, anything that is not supported by the communication stacks available in the consumers or the services).

  • Orchestration, which includes message routing, transformation, aggregation and creation of composite services.

These capabilities are needed in many SOAs, and I think ESBs are a good way to implement them.

ESB products usually perform other capabilities like monitoring, security, logging or endpoint management. These other capabilities are useful only if the ESB mediates in every consumer-service interaction, because otherwise you need also of additional mechanisms different than the ESB.

This is the other meaning of the word ESB: an architectural pattern in which the SOA has three main type of elements: services, service consumers, and the ESB, which is an element independent from the other two and which is the neccessary mediator in any interaction between the other two. To me, to call a product an ESB it must allow to implement this pattern, and this is why I do not see the IONA middleware, which is agent-based and not an independent element, as an ESB.

I think the ESB pattern makes sense in some kind of SOAs (more on that later), but I consider that saying that every SOA has to follow this pattern is plainly wrong. A Service-Oriented Architecture is a way to structure an IT system so that as most logic as possible is implemented in services which can be freely consumed by any other element of the SOA, as long as it knows the service description (mainly, its interface). This vision does not need a intermediate element like an ESB to make these interaction possible, and thus requiring it is just misleading.

As I said, to use an ESB to implement the latter capabilities requires not only the ESB product but also to follow the ESB pattern, but not every SOA has to implement this pattern. If your architecture follows the ESB pattern you can well use the ESB to implement these, but your architecture may not follow that pattern, and thus an ESB would not be useful to implement these capabilities and you will have to use other solution to implement them. However, for the two first capabilities (communications and orchestration), an ESB may be useful in any kind of SOA.

So I am not against the ESB product per se, but more against the ESB pattern being mandatory. I think the ESB pattern makes sense in some cases, but it is not the only way of doing SOAs. Indeed, for many SOAs it is a wrong pattern.

The ESB pattern is actually an integration pattern. The more diverse sources to integrate you have, and the simpler they are, the more useful is the ESB pattern. The extreme ESB pattern requires very basic services which are fully decoupled, i.e. which are never consumers of other service. Then, you integrate them using the ESB, and after that you can have other consumers leveraging the integrated set. Of course, this happens to be the same as the EAI pattern, in which you have existing systems you want to integrate. This is not casual, because ESB products are actually an evolution of EAIs.

But, while an EAI is a solution for integration and only for integration, SOAs are more than this. For example, if you do not have existing systems to integrate (e.g. a new distributed system is created from scratch), an EAI product would be useless. But a SOA would still make full sense, because SOA is just a way to structure functionality inside of a IT system by means of services.

This is a common mistake: many people think that the main value, even the only value of SOA is integration. This is wrong. SOA is more than that. Of course services are a good way to expose the functionality of systems to be integrated, but this is not the only usefulness of services. Besides integration, services are useful as a mean to split functionality in bits and then reuse it.

These people sees Service-Oriented Architecture as an integration architecture, and this is why they see that the only way to create a SOA is by means of the ESB pattern. But this is one flavor of SOA, not all of SOA.

Thus, I see ESBs as a good means to implement managed communications and orchestration when you need them, which is often. This is, if the interaction between consumer A and service B needs managed communications, it is OK to put an ESB in the middle. If you have three services C, D and E and they are often used jointly in a coordinated way, it is perfectly good to use an ESB to create service F which would orchestrate them. If you have service G with an interface but you have consumer H which expects another interface, is is fine to use an ESB to implement H's interface and transform it into G's interface. And so on. Of course you can use also other methods than ESB for that, but ESBs are just fine.

But if you want to log all the messages going through your SOA, or enforce security polices for consumers and services, or monitor activity, or be able to change the location of your services without broking the SOA, an ESB is useful only if you follow the ESB pattern, which may well not suite your SOA. If you have an integration-intensive SOA it may be the best option to follow, but otherwise it is better you look for other ways of implementing these, because it could be negative to force your SOA to follow the ESB pattern.

The bottom line: I see perfectly sound to use an ESB to implement managed communications and orchestration in any kind of SOA. But for other capabilities like monitoring or security, unless the ESB pattern suites your SOA I would rather recommend using other kind of solution, like agent-based middleware or just SOA communication libraries. The more integration needs you have, the more useful is an ESB to implement your SOA.

I think the ESB frenzy is temporary and that eventually everybody will see things this way. But there has been people saying similar things since 2004, and still the ESB is commonly presented as the king of the SOA. However, the implementation of larger and more complex SOAs, and the growing importance of governance and registries and its usage at runtime, may displace the focus from the ESB and change this perception.


Weblog Commenting and Trackback by