Long term evolution over short term expediencyArchitectural clarity over coding efficiencyBusiness Strategy over IT strategy
Drive IT costs in-line with their business valueDrive IT technology selection in-line with the business valueCreate clear upgrade boundaries between different business value areasManage IT based on the different business value areas
The SOA value system is paraphrased below as:
- Business Value. Technology for its own sake should be avoided.
- Strategic Goals. You can't really talk about SOA if you aren't looking at the big picture.
- Intrinsic Interoperability. There's this notion of 'intrinsic interoperability' that needs to be achieved.
- Shared Services. Services are to be designed to be shared (note that reuse is not the termed used here).
- Flexibility is preferred over optimization (typical trade off is flexibility versus efficiency).
- Evolutionary Refinement. SOA doesn't attempt to boil the ocean and an incremental approach is preferred
guiding principles. Here's a quick take for each of the principles:
Respect the social and power structure of the organization. - Sounds like being mindful of Conway's Law.
Recognize that SOA ultimately demands change on many levels. - If you can't convince everyone, then you could fail. This statement seems to run counter to the idea of evolutionary refinement.
The scope of SOA adoption can vary. Keep efforts manageable
and within meaningful boundaries. - Look for low hanging fruit and quick wins to show success.
Products and standards alone will neither give you SOA nor apply
the service orientation paradigm for you. - Don't just buy technology, hire consultants as well.
SOA can be realized through a variety of technologies and standards. - Breaks the bonds completely from WS-*.
Establish a uniform set of enterprise standards and policies based
on industry, de facto, and community standards. - Don't attempt to re-invent the wheel.
Pursue uniformity on the outside while allowing diversity on the inside. - Some kind of notion of polymorphism and encapsulation going on here.
Identify services through collaboration with business and
technology stakeholders. - Don't design in a vacuum, get people involve. What about the consumers, do they have a voice?
Maximize service usage by considering the current and
future scope of utilization. - Plan for change.
Verify that services satisfy business requirements and goals. - Get customer feedback.
Evolve services and their organization in response to real use. - YAGNI principle.
Separate the different aspects of a system that change at different rates. - Probably the more interesting guideline here. Usually, architecture layering is the approach here, however I am unclear if there are other mechanisms to do this. If this is to imply the notion of 'separation of concerns', then I'll be really disappointed.
Reduce implicit dependencies and publish all external dependencies to increase robustness and reduce the impact of change. - One of the better guidelines, a generalization of the idea of the utility of documenting or even exposing for machine consumption a services dependencies.
At every level of abstraction, organize each service around a cohesive
and manageable unit of functionality. - A.K.A. Cohesion.
The paper lays out "Seven Principles of SOA Success". To summarizes the unamed author list these principles:
- Minimize costs of disruption.
- Integrate incremenatally.
- Reduce coding.
- Use industry standards whenever possible.
- Accept the things you cannot change.
- Understand the strategic value.
- Buy - don't build.
It's a pragmatic whitepaper that's worth a quick read. Too many integrations projects fail because of the desire for perfection. A drive towards perfections leads to idealism and creates vision that can easily become unrealistic. That is the crux of the problem, the person that can envision the ideal world is in many cases the person who cannot see reality.
This reminds of an entry I wrote "Ten Fallacies of Software Analysis and Design". Where I outlined several intellectual sink holes which many developers are victim to:
- You can trust everyone
- Universal agreement is possible
- A perfect model of the real world can be created
- Change can be avoided
- Time doesn't exist
- Everyone can respond immediately
- Concurrency can be abstracted away
- Side effects and non-linearity should be removed
- Systems can be proven to be correct
- Implementation details can be hidden
transactions and SOA that is worth taking down:
(a) if you really, really need transactions - don't attempt to do it with Web Services. The CORBAs and the TPMs of this work do that stuff bettter than any amout of new fangled lets-do-it-with-angle-brackets ever will.
(b) Do you really need transactions? There are times when yes, you absolutely do but equally there are times when you ended up needing transactions become of a function centric/object centric design model. Given that distributed transactions are hard and expensive (you cannot design away the network, you can hide it with dollar bills but you cannot design it away), it makes business sense to only use them when you need to.
(c) Automatic rollback? Give me a break. 99% of business process rollback is business logic specific and is most cost effectively done by people who understand the context and can compensate much more intelligently than machines. If you have really simple compensation requirements then congratulations! You are in a minority in my experience.
(d) Systems that have ACID semantics make perfectly good services to expose on an SOA. Like any paradigm, one of the skills of service decomposition is knowing when to stop. A temporal coupling boundary (such as a classic ACID) is a good place to stop.
Lean focuses on the elimination of waste in processes. Agile in comparison is tuned toward practices that adapt efficiently to change. There are though some commonalities between Lean and Agile Development. These specifically are:
- People centric approach
- Empowered teams
- Adaptive planning
- Continuous improvement
The last two bullet points align directly with the SOA Manifesto. For reference:
- Business value over technical strategy
- Strategic goals over project-specific benefits
- Intrinsic interoperability over custom integration
- Shared services over specific-purpose implementations
- Flexibility over optimization
- Evolutionary refinement over pursuit of initial perfection
Lean differs from Agile in that Lean is purported to be designed to scale (see: "Set-based concurrent engineering" and Scaling Lean and Agile Development ):
One of the ideas in lean product development is the notion of set-based concurrent engineering: considering a solution as the intersection of a number of feasible parts, rather than iterating on a bunch of individual "point-based" solutions. This lets several groups work at the same time, as they converge on a solution.
"Haugen in one of the comments pointed me to this gem of an paper. The authors propose any "pragmatic action" as a triple. A triple that involves an intention, an action and a object.
The intentions are propose, accept, reject, declare, query, reply, assert (i.e. Illocutionary). The actions are create, change, cancel, none (i.e. Perlocutionary). The objects are Agreement, EconomicContract, EconomicComitment, and EconomicEvent. The paper is slanted toward supporting business based messaging, however we can take the liberty of generalizing it a bit.
The interesting thing to notice is how the action parts seem to neatly match the verbs found in REST. That is PUT=create, POST=change, DELETE=cancel and GET=none.
With these triples you can build up several business transaction patterns:
* Commercial Transaction Pattern - Requesting activity carries the pragmatic action (Propose, Create, aContract), Responding activity carries the pragmatic action (Accept, Create, aContract), OR Responding activity carries the pragmatic action (Reject, Create, aContract).
* Query/Response Pattern - Requesting activity carries the pragmatic action (Query, None, anyObject) Responding activity carries the pragmatic action (Reply, None, anyObject).
* Request/Response Pattern - Requesting activity carries the pragmatic action (Query, None, anyObject), Responding activity carries the pragmatic action (Reply, None, anyObject).
* Request/Confirm Pattern - Requesting activity carries the pragmatic action (Query, None, aComittment/aContract), Responding activity carries the pragmatic action (Reply, None, aComittment/aContract).
* Information Distribution Pattern - Requesting activity carries the pragmatic action (Assert, None, anyObject), Responding activity carries no pragmatic action.
* Notification Pattern - (Declare, Create, aComittment/aContract)
So, if we adopt REST's existing structure where the action part is contained in the envelope. It like inverting the Speech Act hierarchy (i.e. Pe
these patterns in building what he calls 'alsmost-infinite scalability'.
- Entities are uniquely identified - each entity which represents disjoint data (i.e. no overlap of data between entities) should have a unique key.
- Multiple disjoint scopes of transactional serializability - in other words there are these 'entities' and that you cannot perform atomic transactions across these entities.
- At-Least-Once messaging - that is an application must tolerate message retries and out-of-order arrival of messages.
- Messages are adressed to entities - that is one can't abstract away from the business logic the existence of the unique keys for addresing entities. Addressing however is independent of location.
- Entities manage conversational state per party - that is, to ensure idemptency an entity needs to remember that a message has been previously processed. Furthermore, in a world without atomic transactions, outcomes need to be 'negotiated' using some kind of workflow capability.
- Alternate indexes cannot reside within a single scope of serializability - that is, one can't assume the indices or references to entities can be update atomically. There is the potential that these indices may become out of sync.
- Messaging between Entities are Tentative - that is, entities need to accept some level of uncertainty and that messages that are sent are requests form commitment and may possibly be cancelled.
These are interesting observations that jive quite well with my table of loosely coupled APIs, as well as my previous arguments about distributed transactions and the need for provisional information and contracts. I believe that these patterns can be implemented on top of ReST using in the manne
Amazon, although I'm unaware of his influence on the development of S3, but it's interesting to contrast some of the design principles of that product:
- Decentralization: Use fully decentralized techniques to remove scaling bottlenecks and single points of failure.
- Asynchrony: The system makes progress under all circumstances.
- Autonomy: The system is designed such that individual components can make decisions based on local information.
- Local responsibility: Each individual component is responsible for achieving its consistency; this is never the burden of its peers.
- Controlled concurrency: Operations are designed such that no or limited concurrency control is required.
- Failure tolerant: The system considers the failure of components to be a normal mode of operation, and continues operation with no or minimal interruption.
- Controlled parallelism: Abstractions used in the system are of such granularity that parallelism can be used to improve performance and robustness of recovery or the introduction of new nodes.
- Decompose into small well-understood building blocks: Do not try to provide a single service that does everything for every one, but instead build small components that can be used as building blocks for other services.
- Symmetry: Nodes in the system are identical in terms of functionality, and require no or minimal node-specific configuration to function.
- Simplicity: The system should be made as simple as possible (- but no simpler).
There clearly is an emerging consensus on the architecture of 'almost-infinitely scalable' applications. Furthermore, revealing insights from the experiences of Google, Amazon, Yahoo and eBay are sheding a lot of light on this subject.
Your SOA governance strategy doesn't have to be a long and highly debated exercise, and it doesn't have to be 30 pages long. The idea is to prioritize the various benefits your organization perceives as important reasons to use SOA. Those priorities then drive the foundational principles for service design in building out your SOA. For example, which is more important to your organization: speed, reusability, or agility?
In this context, architecture does not refer to physical or infrastructure architecture. This is not about how the BPEL engine talks to the enterprise service bus (ESB) or where the service registry would sit. This is more about your service meta-architecture or SOA reference architecture, which will define the taxonomies in the services you build and the services themselves. Again, this need not be a laborious task involving multiple architects debating in a room. A simple first step can be to agree on a clearly defined set of classifications that will distinguish your services. Here are a few guiding principles:
What Is SPML?
SPML is an XML-based request response protocol that is used to integrate and interoperate service provisioning requests. The use of SPML is to enable organizations to set up interfaces for Web Services and applications quickly and securely. This is done by letting portals, application servers, and service centers generate provisioning requests in and across organizations.
If you take a typical SOA security stack, SPML satisfies a complementary requirement for authentication, authorization and fine-grained access control. SPML is used for service provisioning whereas the authentication and authorization of data is done through SAML. Fine-grained XML access control is done through XACML.
Service orientation is a paradigm that frames what you do.
Service-oriented architecture (SOA) is a type of architecture
that results from applying service orientation.
We have been applying service orientation to help organizations
consistently deliver sustainable business value, with increased agility
and cost effectiveness, in line with changing business needs.
Through our work we have come to prioritize:
Business value over technical strategy
Strategic goals over project-specific benefits
Intrinsic interoperability over custom integration
Shared services over specific-purpose implementations
Flexibility over optimization
Evolutionary refinement over pursuit of initial perfection
That is, while we value the items on the right, we value the items on the left more.
Successful SOA (i.e., application re-architecture) requires disruption to the status quo. SOA is not simply a matter of deploying new technology and building service interfaces to existing applications; it requires redesign of the application portfolio. And it requires a massive shift in the way IT operates. The small select group of organizations that has seen spectacular gains from SOA did so by treating it as an agent of transformation. In each of these success stories, SOA was just one aspect of the transformation effort. And here’s the secret to success: SOA needs to be part of something bigger. If it isn’t, then you need to ask yourself why you’ve been doing it.
The latest shiny new technology will not make things better. Incremental integration projects will not lead to significantly reduced costs and increased agility. If you want spectacular gains, then you need to make a spectacular commitment to change. Like Bechtel. It’s interesting that the Bechtel story doesn’t even use the term “SOA”—it just talks about services.