The Rise of the Semantic Contract

October 9, 2008

This is my stake in the ground for now. SOA in the market-place places total emphasis on 2 things. Web Services as a basis for communication and Re-use as a basis for convincing the boss to put some cash into your middleware bunker..no…play-pen…err…seat of learning.

In addition the militant splinter-groups of the new-wave of RESTafarians (of whom I am an empathising skeptic on this specific ūüôā point about service specification) call for the death of WSDL and the reliance on (WSDL-lite) WADL in the case of the less extremist, but plain-old inference from sample instance documents in the case of the hard-core….

I am finding myself sailing down the no-mans land between these two polarised viewpoints, and see the need for specification in the more complex end of the interface spectrum, but similarly don’t see how specifications help when decoding the specification is harder than inferring from samples when interface contracts are relatively intuitive. So there we have a basic mental picture of my map of¬† the universe.

Now I’m getting to the point.

I’m now convinced that SOA’s push for re-use established through WSDL everywhere, but equally the more recent RESTafarian voices relating to unspecification both have flaws when we are attempting to open up a generalised interface into a service endpoint capable of dealing with a range of entity variants (say product types for example).

My view here is that in the SOA landscape, the static and limited semantic capability of XMLSchema, and in the RESTian lanscape, the inability of humans to infer correctness without a large number of complex instance document snapshots, leads me to the conclusion that there is a vast, yawning, gaping, chasm of understanding in what constitutes an effective contract in locking down the permissible value permutations – aka the semantic contract.

I’ve seen MS-Word. I’ve seen MS-Excel. I’ve seen bleeding-eyes-on-5-hour-conference-calls-relating-to-who-means-what-when-we-say-customer, and best of all I’ve seen hardwired logic constructed in stove-pipes behind re-usable interfaces, aka lipstick on the pig.

I reckon the semantic contract – the contract locking down the permissible instances is far more important than the outer structural contract who’s value decays as the level of re-use and inherent complexity of the interface increases. In addition there are likely to be multiple iterations/increments of a semantic contract within the context of a structural contract as service functionality is incremented over successive iterations – adding product support incrementally to an ordering service of example. This leads to to the notion of the cable cross-section:

In the SOA context…WSDL drives tooling to abstract me from the structural contract. But the formation of the semantic contract as the expression of what the provider is willing to service via that re-usable and loose structural contract is the key to effective integration.

If we don’t pay this enough respect we’ll be using our system testing to mop-up the simple, avoidable instance-data related problems that could be easily avoided if we’d formalised the semantic contract earlier in the development lifecycle…

Powered by Qumana

Advertisements

Enterprise Integration: Don’t Look Down !

July 4, 2008

Can SOA truly be successful if service consumers have to be technology consumers? The service layer is supposed to insulate us from the technical complexities and dependencies of the enabling technology, but I see more and more the technology being the centre of attention.

The promise of Web Serivces whilst standardising on the logical notion of integration,¬† has embroiled us in a complexity relating not to the ‘act’ of exchanging documents, but instead relating to the diversity within the Enterprise, the various technologies and tools used across a widely disparate ecosystem, and debating the finer details of which interpretations of which standards we want to use.

More significantly the large deployed base of messaging software, and the service endpoints exposed to MQ or JMS endpoints are left out of the handle-cranking associated with the synchronous style of endpoint. As such – a SOA layering ‘consistency’ across such a diverse ecosystem is a myth in my experience. We’re still struggling to find the SOA ‘blue-touch-paper’ despite all of the top-down justification and policy.

I believe that until we push the technology further down towards the network such that it becomes irrelevant to the service consumer, and raise the service interaction higher up in terms of decoupling the ‘interaction’ from the ‘technology’ we are going to struggle to not only justify the benefit of service orientation, but more significantly we’ll continue to struggle to justify the inevitable rework and technical implications of that service orientation in mandating conformance to brittle and transient technical standards.

I’m going to explore an approach to doing this – by encapsulating middleware facilities as RESTian resources, and then looking at the bindings between WSDL generated stubs and these infrastructure resources…effectively removing technologies (apart from the obvious RESTian implications) from the invocation of a web-service. Various header indicators can flex QoS expectations in the service invocation (i.e. synch or asynch, timeouts, exception sinks etc) but that has no relationship to any given protocol or infrastructure type. Furthermore, the existence of such a set of ‘resource primitives’ would enable direct interaction where WSDL-based integration does not yet exist…where I resolve, send, receive and validate though direct interaction with RESTian services from any style of client-side application.

This is motivated purely by the belief that, much like the chap in the picture, our focus is on the endpoint and not what lies beneath…


REST: Logical, Cosmetic or Fundamental

June 24, 2008

In my experience the successful application of the RESTful architectural style to any given software integration opportunity, requires an appreciation of the levels of RESTfulness gained from the relative levels of effort expended. I will attempt to explain my interpretation of these and identify where I see positive an negative implications:

Logical REST

The approach where the interactions between agents are ‘designed’ as resource-oriented, based on a finite set of well-defined methods analagous to the RFC2616/HTTP ‘verbs’, or more generally a CRUD scheme. This kind of approach achieves common-ground in the design-space, but is not followed-through with a strict adherence to the RFC2616/HTTP protocol as a framework for implementation, possibly using a proprietary xDBC stack, language-specific binary protocol or heavy SOAP stack. As such we see a logical resource-oriented perspective but with none of the true benefits truly embracing the web as a foundation.

Advantages

  • Achieve uniformity in resource/interface design.
  • Establish a common inter-domain design philosophy.

Disadvantages

  • Not addressing the heterogeneous middleware friction.
  • Stopping short of realising the full value of REST.
  • Little or no departure from traditional CRUD data-access.

Cosmetic REST

This is possibly the most common category where a Logical REST design is implemented within a RFC2616/HTTP framework, exposing concrete resource addresses on well-defined http URI’s in a web-compatible manner. However, the cosmetic dimension relates to the fact that these RESTful endpoints are thin, ‘cosmetic’ wrappers over traditional RPC oriented service endpoints, effectively acting as proxies. As such Cosmetic REST is of benefit to clients who have less intellectual and technical friction in consuming a service, but do little to thin-out the general integration sprawl within the domain of the service-provider. As such the addition of public RESTful to private RESTless bridging increases the integration burden of the provider.

Advantages

  • Creation of real RESTful service endpoints removing technical friction from clients when consuming service.
  • Unhook clients from internal integration dependencies by providing a uniform, intuituve interface.

Disadvantages

  • Increases the integration burden of the provider.
  • New facade on existing brittle integration points.
  • No increase in productivity gained from supporting simplified exposure.

Fundamental REST

This approach is a full application of the architectural style in a deeper sense than the cosmetic use of the HTTP protocol in exposing interfaces. Resource oriented design of a system is a significant departure from simply exposing functionality or information services from a complex machine using some arbitrary exposure/protocol dialect. Instead resource orientation inverts the ‘functional’ exposure of traditional RPC styles of integration by facilitating an information-first (not functionality first) interaction between a client and a system. In simple terms your interaction with a true RESTful system is through the manipulation of these resources, where creation, updating or deletion of resources or resource-atributes has explicit (or implicit) behaioural contracts which will conclude within a stateless atomic context. This completely decouples a client from the internal workings, dialects, processes, etc of a traditional approach to opening out the functions within a ‘RESTless’ system not based on the resource orientated architectural style.

Advantages

  • Cleaner relationship between RESTful exposure and implementation of underlying system.
  • Reduces the ‘thickness’ of interation logic sitting between the network addressable endpoint and the implementation.
  • Integrate clients with your information and not your process and/or functional granularity.

Disadvantages

  • Despite the RESTful or resource-oriented architetural style being synonymous with the web, it is still perceived as a ‘new approach’ in the context of internal enterprise business systems.
  • A more significant mind-shift is required before this appreciated
  • Still little hard evidence of the positive implications of resource orientation over traditional approachs¬† –¬† it just ‘feels’ like the right thing to do.

In my experience to date, for anything other than a green-field web-application which may go directly to a Fundamental REST approach, the larger integration challenges associated with service enablement of a more complex enterprise or enterprise system will ultimately start life, as we have, in a Cosmetic RESTful approach – most likely resulting from the combination unmovable silo’s of historical investment, as well as the limited impact of a Cosmetic or Facade based approach when breaking new ground.¬† In such cases the cost-effectiveness of REST and resource-orientation do become hamstrung, purely because to exploit REST you have to build more layers of mediation initially…until the confidence in REST increases to a point at which true, or more Fundamental REST can be driven from the edge of the enterprise, down into the silos.


Making REST a reality…

February 19, 2008

2008_02190006_small1.jpg

The price of being RESTful (I daren’t say ‘simple’) in a complex world…where did that lot come from? (a reality-desk-cam snapshot after a quick tidy-up).


It Might Be Just Me….But…

February 19, 2008

So I see REST as a re-claiming of the common-sense ingredient in the whole application integration landscape. I see the how the use and re-use of the web as a means of linking not only customers with business, but with any participant in a complex transaction, a fantastically liberating vision within the Enterprise. I see how REST combined with the Enterprise has the potential to weaken the strangulation of vendors, and their crippling commercial strategies on the progress of the Enterprise. And I see a gradual and accelerating simplification of how we wire together our Enterprise IT assets using web principles, as opposed to the WS/MOM/EAI stacks which have prospered under the false¬†“SOA==WS==Fortune-and-Glory” banners waved by all the vendors over recent years…..

So….when I see a major software vendor claiming that their proprietary, highly expensive product-line will now support REST, why do I suddenly feel my head begin to shake from side-to-side slowly….whilst grinning from ear to ear with that sort of ‘I’ve been here before and it hurt‘ kind of grin?

Disecting this statement further (when my head had stopped shaking) I find that what they actually mean is not ‘supporting the exploitation of REST principles in removing unjustifiable cost and proprietary¬†stranglehold partnership¬†from the Enterprise by rediscovering the power and¬†boundless¬†coolness of¬†HTTP, and liberating some self-confidence to use it properly….¬†‘

But instead it actually means ‘you can still use our highly expensive, proprietary technology stack, therefore allowing us to propagate our lock-in¬†strategic-alliance and stranglehold partnership, because we’ve added a http management interface that looks RESTful‘.

Slapping a RESTful management interface on my fridge aint gonna help me simplify my Enterprise is it now? ūüė¶


Implosion of REST Frameworks?

February 19, 2008

Applying REST principles to a complex Enterprise is harder than applying them to a photo-database or a blog-site (aka skinned database). No points for that one. Why do it? Well treating an Enterprise as a set of ‘resources’ with representations and behavioural contracts applied to attributes manipulated through a uniform, ubiquitous interface is a liberating experience on the design-slab. However when we come to take real steps we are beset by 2 key obstacles:

  1. Resource representations are abstractions of a complex information architecture underpinned by many diverse and evolving IT assets, so we do not have atomic control over the assembly of resource representations nor do we have atomic control over key-mastering and complex updates.
  2. The resource Namespace needs to support hierarchy to ensure we can correctly structure context around certain resource-types who’s representations may be context-aware. As such depending on who is looking at what, there is a complex resolution of the¬†public¬†namespace to a potentially complex EAI landscape within the black-box.

Are these problems insurmountable? Well I don’t believe so, but the resulting shape of the framework I’d use to achieve this is significantly different to the emerging frameworks (i.e. JAX-RS/JSR311, Restlet, etc) geared towards resource-enabling stuff. Why?

The primary reason why I seem to be diverging from the direction being taken by the framework guys, is simply that (…note to self…I’m not officially recognising options which would require me to acknowledge that I’m being stoopid ūüôā ) I need more dynamic processing in the mapping of a requested URI to a POJO/Controller. What does this mean?

Well – based on the 2 points I outlined at the start of this article the inspection, validation and refinement of my inbound URI’s are managed within a single control step, fed with meta-data and context info. Similarly, as I resolve a canonical resource (i.e. one which may be reached through a range of public namespace paths) I need to work out how best to ‘go and execute’ the require data-acquisition or functional RPC steps to implement the resource operation. Again – another control step fed with meta-data and context info.

What I end up with, is a processing chain from HTTP request through to Enterprise resource, where the namespace, URI resolution and method variations are simply parameters driving a more complex, dynamic integration broker. As I DRY up my code, I begin to refactor away from having explicit resource controllers be that at a class-oriented or annotation-based level.

Ultimately my ideal ‘REST’ framework is one which enables me to data-drive not only the production of a resource facade, but the linkages between those resources and complex integration/aggregation transactions, therefore imploding the more explicit genre of ‘REST framework’ in favour of a declarative alterative.

At this time I’m prototyping with SpringMVC Servlet/Interceptors and also raw Servlet/Filter chains, both of which are configured to have only 1 front-controller, and a series of processing steps as the inbound request is boiled down to a back-end transaction. The bulk of the manifestation of REST-oriented logic is meta-data consulted by the processing chain within the context of a user-initiated operation.

This is what I mean by Implosion of REST frameworks…


Ruby/Rails URI Patterns and JSR311

January 10, 2008

I had been working with a rails project mapping a uniform REST interace onto complex resource representations assembled at runtime, and as such no object/relational mapping was used to pull information from a local datasource.¬† Instead I use ‘virtual’ resource representations which are fulfilled on-demand, within synch or asynch interactions, from back-end complex back-end data sources and interfaces.

As such the primary Rails convention of mapping resource¬†URI’s to controllers and actions, and through an object/relational layer to a datasource is effectively useless in my scenario.¬†Instead of mapping resource to a datasource, I instead have to¬† map URI’s to a generic controller, still supporting the GET, POST,¬†PUT, DELETE semantics, but¬†essentially data-driving a dynamic integration tier with the resource context.

So¬†if I were¬†resource-enabling a database with¬†¬†2 tables, customers and orders, then I’d use the following routes.rb entries (along with a rails resource scaffold) to build my URI to datasource relationships.

map.resources :customers
map.resources :orders

This would route requests for http://domain:port/customers to the Customers controller, and route the http://domain:port/orders to the Orders controller, where each would sit across O/R mappings to specific tables in my database. Nice and easy !

However in my world the Customers resource and the Orders resource are virtual within the front-end runtime, and must be instantiated by using a dynamic mapping algorithm to initiate back-end integration transactions over a range of mechanisms to instantiate the correct representations for each resource type (focusing on GETS’s for this sake of this example). As such it makes¬† sense for me to have a single VirtualResource Controller, which uses request context information to infer the correct representation algorithm. Rails has a neat way of enabling this in the routing table:

map.resources :operations, :controller=>"virtualresources"
map.resources :customers, :controller=>"virtualresources"

This enables me to pipe a range of URI patterns into my virtual handler and manage a lot of complexity through very few generalised routines.

At this point I’m considering the migration across to Jersey on Glassfish – using the JSR311 annotation scheme for this kind of URI pattern matching.

 @UriTemplate("widgets/{id}")
 public class Widget {
 ...
 }

Only problem is – at the moment I see no way of assigning multiple URI patterns to a single Virtual Resource class in the way I have with Rails. Only one URI pattern is enabled for each POJO class – which would mean I’d have to create many proxy classes into my virtual resource handler? I’m gonna have to dig a little deeper here but hope this kind of flexibility and learning from the Rails community will be factored into the JSR spec?