I have been aware for some time that in general, when it comes to the integration of two or more software agents, we seem to have a disproportionate amount of effort across the two critical business integration dimensions – syntax and semantics. The vast amount of modelling and design effort is targeted at the syntax end with DTD, WSDL, XML Schema, along with validating XML parsers across a range of development platforms ensuring we can sniff a missing element at 100 yards. The semantic dimension is to all intents and purposes cast-aside as a mopping up exercise which trickles down the food-chain until it sticks to some poor developer locked in a code-vacuum who’s never even considered what the information actually means as it pings it’s way through his n-th generation-optimised integration stack in 0.002 msecs with 99.999999% reliability !
We’ve all been there…after the integration designs have been signed off….where the XML schemas do a great job of locking down the envelopes through which we then pass combinations of xs:any subtrees and batches of name-value pairs, at which point the real fun starts. Out comes the spreadsheet, with 4 columns:
- Source System Field Name
- Common Model Field Name
- Destination System Field Name
We then fast-forward a couple of months, by which time the integrators and the systems teams hate each other, and we’re running into the max-row limit in Microsoft Excel – but worry not, we managed to get all the mappings sorted….or did we? We may have filled out the syntactical relationships between the elements (if we’re very luck) but we generally fail miserably to capture and manage the meta-data which will enable cleaner integration over subsequent iterations.
It is usually the free-form column that contains the really high-value meta-data, generally in free-form, but a rich vein of valuable constraints and business-rules which rarely manifest higher up the modelling and design food-chain for a number of reasons. It is this meta-data, captured as a by-product of the mapping activity, which we must capture, formalise, version control and publish for subsequent users/developers of the particular interface we’re constructing. We have UML models and GUI mappers for the easy stuff….but at the business-end there are very few semantic integration frameworks – well only one that I’m aware of (Progress DXSI) as a maturing capability offering a lifecycle process as an alternative the spreasheet…..and it enables us to extend our design governance process beyond the creation of static artefacts commonly associated with integration designs. I once referred to this as ‘the year 2 problem’ in a SOA transformation – only really coming to the fore when we begin to work on v1.x interations of our baseline services, but only IF we care about resue and cost-reduction in the design process. Based on the lack of attention in this area I think that assertion may prove a little optimistic…