SOA : A Destination or a Journey?

December 22, 2007

I’m having a random interruption and I feel strongly enough about this point to post it in isolation.

  • Enterprise Architecture and SOA are not inextricably linked – I can form an EA vision which may rely on an implementation approach which is not SOA. SOA is a particular approach I must adopt if I have a dynamic environment, where I must constantly evolve my products, processes and IT stack, and where my agility is my market advantage. SOA is not mandatory.
  • Perfecting the process of applying Service Oriented Architecture principles to the Enterprise is the ‘goal’. I am emphasising SOA in the Enterprise as opposed to SOA at an application level which is trivial by comparison. Getting agreed SOA principles into the organisational DNA is the holy-grail. Once our vision and our ability to execute is in lock-step, what do we care about changes in the vision?
  • ‘Finishing’ the Enterprise SOA transformation is NOT the goal…if it were the goal then why all the focus on re-use, agility, insulation from wolverines and all the defensive positioning around SOA? The goal is to equip the enteprise with the IT capability to support it’s evolving business model, and if there was a finite end-state then I would question the value of such a large investment.
  • There may be a series of functional snapshots at which point the SOA is fulfilling transient business needs, but the underlying SOA infrastructure continues to evolve, refine and take on additional business requirements as a matter of course.
  • Enabling the business transformation, and ensuring the IT estate is capable of continually adapting is the goal and the one and only reason we should be embarking on large scale SOA programmes.

Pack for the journey, not the destination…you’re intention is to travel…oh and make sure you don’t delegate the navigation (or worse – pay someone by the hour to do it for you!!!)…keep hold of the map!”

ESB Part 2 : The Shapeshifter…

December 21, 2007

So the obvious, and much publicised, question one asks when hearing about the Enterprise Service Bus, is what is it? Is it a style, is it a thing, or is it #*%!!

I have a particular take on this borne out of my own experience in having to justify such a thing within a large scale enterprise architecture programme. As per my earlier post on SOA==WS?, one’s perspective on SOA has a huge bearing how the ESB shapeshifter can be manifested within your problem-space.

 In a traditional EAI landscape, we centralise the integration problem. This means that we enable the interacting systems to utilise local dialects when firing requests into the EAI space. The EAI brokers then deal with the transformation from external dialect, through the common, canonical dialect and into the external outbound dialects of the target systems (I am generalising in a big way however….). Difficult business integration problems are pushed away from the endpoints into the magic-in-the-middle, and this is where much of the EAI bad-press has come from, simply because it trades in peoples pain ! Would we need it if integration was easy ?

 So my primary driver when looking to evolve my enterprise integration roadmap to support the unfolding SOA transformation, was to decentralise the pain and give it back to the owners. Not just to reverse the trend, but to drive convention and middleware infrastructure templates back to the edges of the relevant IT domains such that the runtime becomes federated.

This federation has an implicit but significant reliance upon consistency and contracts at the endpoints, whilst opening out a consistent, scalable messaging conduit (finally!) between the domain brokers. Standing back from this – we have clients and servers adderssing the transformation in/out of service contracts at the edge of their domain, with common representation, messaging and infrastructure management in the space between. The EAI Bus architecture was taking shape.

The most significant thing in my opinion was the reduction in diversity by prescribing inter-domain standards. Finally the EAI brokers didn’t need the full range of COTS and Technology adapters they were usually encumbered with, nor did they require the high-end process-management abstractions, and as a result the technical requirements on these components reduced dramatically.

Over time, the movement of endpoint-specific transformation, standard messaging and document convention and runtime independence to the endpoints became analagous with the vendorised ESB concept which came along subsequently.

 However – my experience places the term ESB (and with major Emphasis on the ‘E’) as a scalable Enterprise framework, incorporating disciplined distribution of the integration problem space, and the consistency evolving int the messaging backbone.

I do see, and understand alternative perspectives (and I will use the term ‘eSB’ where the casing of the ‘e’ implies the technology being used in a less than Enteprise capacity. Where we see a the (systems not Enteprise) architecture incorporating a single messaging and transformation broker – supporting service contracts at the edges…implying the clients and servers are already emitting/consuming the expected dialect. I’m less convinced by this model and the shades of grey between this and my own perspective – purely because I am focused on the Enterprise ‘E’ in ESB as opposed to the ‘e’ in eSB…

Debate (SOA==WS)or(SOA!=WS)or((SOA==WS)and(SOA!=WS))

December 21, 2007

I see regular debates around the relationship between SOA and technology, with the bone of contention the assertion that SOA==WS. Whilst I empathise with all of the justifiable positions offered across the divide, I sort of see this as a crazy argument.

Why? Well in simple terms just as ‘beauty is in the eye of the beholder’, then I argue that ‘SOA is in the eye of the beholder’ too. One’s perspective relating to the term architecture has a huge bearing on what we then regard as service orientation.

Zooming in as far as we can, we have the ‘SOA’ project relating to wrapping a legacy application or databse such that legacy technology dependencies are unhooked from the consumers of said services. This aligns to a systems/software architecture perspective.

Zooming out as far as we can, we have the ‘SOA’ programme running hand-in-hand with Business Transformation and Enterprise Architecture programmes, whereing the service orientation relates to the process/functional decomposition, placing certain ‘service’ obligations on various traditional silo’s as the architecture becomes more open. This aligns to an enterprise architecture perspective.

In each case we have a radically different scope and granularity to the term SOA, and as a result one has more of an emphasis on a technical interpretion and a reasonable fit with the SOA==WS. However the enterprise SOA space is further removed from the technology and justifies the SOA!=WS.

My conclusion is that SOA as such an amporphous term (like architecture itself) must first be qualified before engaging in such a debate….

REST – Am I moving from ruby to java…?

December 21, 2007

So I’ve concluded some deep investigations around RESTful service applied to the big nasty Enterprise. I’ve been easing myself back into hacksville with Windows/Ruby/Rails/MySQL and have had an altogeher positive experience. However I’m now at a point of considering my next steps – do I go forward on Ruby/Rails or do I take this opportunity to hop onto an alternative framework.

The answer to this relates to the nature of the challenge in applying REST to the Enteprise as opposed to RESTifying a database or application instance. In the former the primary challenge relates to the formation of resoruces from potentially complex components – not atomic records or aggregations of information from a local-datasource (I’m using a trivial scenarion here as an example – not disresepcting restful applications !!).

 I have hit problems with managing synchronous and asynchronous RESTful interactions within rails to date, and these challenges are purely down to the complex linkage beween a RESTful presentation and a complex implementation landscape.

Whilst this kind or perspective may be off the mainstream from a rails-applcations perspective, it’s core to my world – and as such I benefit less from the excellent front-end rails conventions, given I’m assembling resource representations which don’t exist in an atomic sense, in a local DB.

So the key issues I now face in my next iteration (and the Rails/Java JSR311 question) are:

  1. I need a more seamless mechanism for managing undeterministic concurrency (by virtue of possibly long-running resource operations), and where dynamic threading seems cleaner than pre-defining a mongrel cluster for a known concurrency limit up front – along with a fron-end load-balancer configuration.
  2.  I need to introduce efficient, selective asynchronous interactions based around the 202-Accepted response, enabling the resource server to work on an operation whilst pushing a client away with a URI.
  3. XML Schema and validation support. I understand, to some extent, the position offered by the Ruby/REXML community in that XML Schema is over complicated and useless….and not worth implementing within the REXML library. However XML Schema is a mainstream currency in my integration (Enterprise) landscape and as such I have to deal with it. The lack of support in ruby has hampered my efforts to date.

I stress again that this is not a gripe about ruby/rails by any means – simply an observation that my Enterprise/REST context is emphasising certain integration challenges which may not be front-and-centre in the emerging waves of RESTful applications successfully launching on Rails.

So I’m looking at Jersey (JSR-311 Reference Implementation) and Noelios Restlet as options for my next leap. Both options will cover all 3 of my pinch points (I believe), and the thing I’m unsure about is the extent to which I’ll be exposed on the presentation stuff which the rails convention eats for lunch….

An interesting junction….

ESB Part 1 : The Sales-Tool or the Strategic-Backbone

December 19, 2007

The Enterprise Service Bus (ESB) is a much maligned, scorned, mocked and generally over-vendorised concept which has resulted in it’s value being shrouded in the usual layers of skepticism. As a former Integration Architect within a large scale Enterprise SOA transformation program, I have spent a huge amount of time looking at the optimal formation of hundreds of middleware silo’s as the strategic backplane of the Enterprise SOA service-tier.

My interest in ESB started way before the ESB hit the hype-cycle, whilst deputising for an amazingly talented Integration Architect, and initially we were simply looking for an architectural model that would allow controlled decentralisation of the middleware bottleneck responsible for strangling (whilst bankrupting) the wider mobilisation of SOA implementation across our enterprise. The initial term we used, coming from a hub-centric EAI landscape, was EAI Bus. I am referring to an EAI hub proliferation on a major scale in terms of cost and numbers. The net result was the spreading loss-of-control of the business systems to achieve integration on anything other than a tactical basis.

 So the EAI Bus referred to the creation of a standardised inter-hub protocol over MOM infrastructure effectively extending the remit of any domain-hub (be that product-line, business-unit, or whatever….) into forming an component part of a larger, addressable enterprise backplane. The motivations were:

  1. Establish a common inter-domain integration standard, leveraging existing investment in domain EAI (BEA) and Enterprise MOM (Webpshere MQ, BEA JMS) technologies.
  2. Extend addressability across the federated hub namespace – effectively defining the common rail across which services (either strategic SOA assets or legacy EAI endpoints) can be located.
  3. Empower the IT domains at the edges, who are seeking to integrate on a strategic service-basis, but avoid the inter-domain EAI complexities in favour or working through a local integration unit where possible.

The decentralisation of the integration platform infrastructure and it’s dissemination into the service provider/consumer endpoints was the fundmantal goal. This was an evolution from our EAI reality, and I stress this commenced before the ESB band-wagon kicked in. I’ll be posing additional stages of our ESB evolution story.

The key point I can offer here is that WE knew what we were looking for, prior to the ESB arriving so we had a strong vision and strong motivation that allowed us to refactor existing infrastructure as opposed to re-investing in dreams.

Rails is now ‘The Rail’

December 19, 2007

Right! That’s it I’ve decided to make a stand. Considering my options I have ruled out hunger-strike and other forms of protest that may impinge my comfort zone – which didn’t leave many options at all really. However I have now rebranded Ruby on Rails as ‘The Rail’ based my disillusionment with it’s lack of thread-safety and concurrency. There ! It’s out in the open.  

Let me explain :

(Ruby + Rails) == Fantastic – Concurrency  == The Rail

Let’s just call this tough love!

Rails: Love/Hate

December 19, 2007

I just realised that I’m split right down the middle when it comes to my opinions on Rails (on Webrick or Mongrel). It’s Ruby affiliation and it’s convention based generator framework is simple elegant and excellent – to the extent that it’s eased me back into the hacking saddle after a 7 year trip around the management wilderness – bereft of the simple ‘truth’ that code brings to one’s life. That aside – my love of rails is perpetually fire-bombed by knowledge that however you want to cut it, it’s single-threaded ! Yes that’s right folks…only one of your controller actions will ever be in operation at any one time ! There are, however,  tentative claims that if you hold your breath, unset a few cryptic, undocumented booleans, then if you don’t whilstle on a Tuesday, it might just let you add some concurrency….so that’s a no then!

I want to love rails more…..but it just won’t let me. You see the fact that it is a single threaded process, taking one request at a time from the underlying Ruby socket stack, sort of makes me start wondering about how such a thing could have come into existence? The concurrency bit is where I’d generally start when developing stuff which is destined to serve clients. Not knocking rails overall, but until this thorn is removed from my side, I’m never gonna find true rails-love!

Currently, server concurrency is achieved by sitting a load-balancer across a group of explicitly created rails processes (mongrel_cluser for example)…..but whilst that is an entirely serviceable concept, it just errodes my perception of this slick framework.  I’m just in disbelief that up to rails version 2.0.1 and mongrel version 1.1.1 it’s still single threaded as a result of not being thread-safe ?!