SOA Manifesto Value Statement Critique

Looks like the SOA Manifesto went public recently. I wasn’t invited to participate, so what does one do when not invited to a party? It’s quite obvious—you tell everyone how bad the party was, right? Ahh, High School memories revisited… Any case, I commend the community for coming together to vet through definitions, principles, and value statements (Heck, Even the SOA is Dead blogger was there, which I find really ironic.) However, I feel a few key points were missed and below are my critiques of the 6 value statements. Keep in mind that my critiques aren’t intended to poke holes in the value statements just due to my lack of involvement; rather, I truly found the value statement to be confusing, not accurate, and a few that are a lot broader than just SOA. I tried to put myself in the shoes of a SOA newbie who is using this to grasp SOA for the first time, and therefore felt obligated to point out areas that could use improvement. I truly hope this is constructive and maybe we see a 2nd version of the manifesto that incorporates this feedback (how Agile would that be?). Please call me and I’m more than willing to help. I love parties!

Value Statement Critique

#1 Business value over technical strategy. This was easily my favorite value statement. In my own Blog, I called it Business Visibility and Understanding over IT Priorities. Clearly, everyone must agree that our goal is to create business solutions. Conclusion: Great statement! The business does always come first! That’s why we’re here!

#2 Strategic goals over project-specific benefits. I am not sure I can agree with this value statement in the fact that I don’t always believe that strategic goals trump project specific benefits. My feeling is if you take this philosophy, you may never get your SOA project started or out of the gates. My rationale is that in order to get SOA funded and jumpstarted, you need to produce something that is demonstratable and appealing to the business, in a very rapid implementation. How else do you get initial buy-in? My feeling is that (1) this value statement is more academic than practical (2) this value statement is also a lot broader than just SOA…So I am compelled to proclaim, Isn't this what the original Agile manifesto is supposed to avoid-- academic principles that are difficult to follow in practice and too abstract to actually leverage?? I think there could have been a more SOA-specific statement in here, while this is more Enterprise Architecture, touchy-feely. Conclusion: I love the idea of everything being strategic (we don’t like tactical), but not sure this will statement work (especially in this economic climate!). Isn’t this what made SOA Dead? Now, do I think one should ignore strategy? No, there is certainly an overarching strategy and vision to work towards something greater and more holistic along the SOA Roadmap; however, I do think SOA needs to start from the ground-up to gain proper momentum and business-level buy-in and the only way to accomplish this is through incremental project implementations that have a discrete goal, are demonstrable, and appealing to the business.

# 3 Intrinsic interoperability over custom integration. I have mixed emotions on this value statement. I believe interoperability is one of the leading principles of SOA, but I also think it’s a bit confusing to have that value prioritized over "custom integration". I don’t think we are comparing Apples to Apples here. I think a better wording of this statement would have been "interoperability over individual silo's"...in my blog I stated "integrated systems, processes, and people over individual silo's". The problem with the "custom integration" wording in this statement is most legacy systems require custom integration in order to make them interoperable! Not everything is component-driven, plug and play or "configuration driven design", which to me are the opposite of "custom integration". I’ve seen a lot of custom integration in SOA, and it was required for the proper transformation of the system. Conclusion: I like the first part of the statement (Interoperability rules!), but find the second part to be inaccurate and confusing.

#4: Shared services over specific-purpose implementations. I do like the concept of Shared Services, as I find re-usability one of the leading principles in SOA. Again, I'm not in love with the 2nd half of this statement, as I'm not sure it is accurate. I feel very strongly that there are many specific-purpose objects in SOA architecture and required throughout the implementations and design. Not every service is going to be a Shared Service or a Business Service. For example, data services or IT services are usually very specific to the system they are interfacing with or rule they are enforcing. Take any data service that is part of the CRUD model on a legacy database—is that a shared service or a specific purpose implementation? Conclusion: I like the idea and whish everything could be reusable, but don’t want to undervalue services that are not!

#5 Flexibility over optimization. This statement gave me the most heartburn. Simply put, I don't agree with this statement. I believe flexibility and optimization are mutually exclusive and therefore one should not be valued over the other. How can you value one over the other? It’s been very clear to me that some services could be static but value optimization more than flexibility. For example,
A high speed transaction in a Financial Services business process would value optimization much more than flexibility. I feel a more appropriate wording of this statement would have been “Flexibility over rigor”. This I can agree with that agility and adaptability are leading principles. But, so is business process optimization. Let’s not forget software engineers, that there is always a trade-off of performance vs. agility in software programming and at the end of the day, some applications (and services) need more speed and acceleration over flexibility and agility. It is always a case-by-case basis. Conclusion: Misleading prioritization. Would really like to see this re-worded and re-released to clarify.

#6 Evolutionary refinement over pursuit of initial perfection I like this statement, as it enforces the iterative nature of "Agile lifecycle and management" and leads one to believe that an early feedback cycle will be incorporated. However, I have to ask, is this principle really specific to SOA? Isn't this just a re-hash of Agile Principle #12? Conclusion: Love this value statement, as I think iterative, early feedback is the way to go, but I think this statement applies to more than just SOA-- any project being implemented should follow this statement, and therefore I diminish the value of this statement only because I feel we heard this before in the Agile Manifesto.

Flying in the Clouds requires many levels of TRUST

I have been fortunate to present introductory concepts and practices in Cloud Computing at 3 different conferences of late. The audience is very active, usually about 50 attendees in the room. However, it's very interesting that every time I poll the audience on who has used cloud services or a cloud provider, I usually get anywhere from 0-1 hands raised. It seems most folks are still feeling out cloud and a lot of it has to do with an overall lack of trust.

I receive a lot of questions on security and data ownership concerns throughout my presentations, but one in particular struck me as a great discussion point. Right when I was building confidence that I had all the answers to everything anyone wanted to know about Cloud, a question got asked that really got me thinking more about cloud and the requirement for TRUST. This particular person simply asked: “What do you do to prevent Cloud employees, particular Administrators with System level access, from jeopardizing or leaking your company’s sensitive information”? It’s a very pertinent question, and although there is no silver bullet solution, the answer ultimately relies on TRUST. If you are going to implement in the Cloud, you must have multiple levels of TRUST. After all, you’re putting protected (and potentially sensitive) company information in the hands of others, and that in itself takes a lot of TRUST. The answer to the questions asked-- you need to trust your provider just the same as you trust your employees (Do you trust your employees?) It's important to follow some of the practices below to help solidify trust in your cloud provider:

(1) Choose your cloud providing vendor wisely. If you don’t TRUST your cloud provider, then you won’t sleep at night. If data ownership and security will bother you to no avail, then you might not be ready for the cloud and using a provider. You must pick a vendor you TRUST to protect your must sensitive information. Do you TRUST your spouse? Yes, of course! This same level of TRUST must exist for your Cloud provider; otherwise you could be doomed for an early divorce. Make sure to meet your Cloud Provider face to face so that you know them well. Interview your Cloud provider. Interview your Cloud provider’s employees. Treat your provider’s employees it as if you’re hiring their employees, because they are now part of your “team”. You need to equally TRUST them as if they were your own, because the will have the same access and privileges. Make sure you have their contact info, background check results, and overall level of confidence from having met them face-to-face.
(2) Iron clad contracts. Template NDA’s are not sufficient for this new paradigm that cloud computing presents (using other people's assets). Your legal team may not be ready for cloud because legal doesn’t TRUST anyone...sorry legal! Talk to legal and explain the model, so that the contracts and non-disclosures provide adequate protection for the worst possible failure (such as sensitive data leak). This is your insurance plan, so make sure you cover your bases and have contractual stipulations to protect your cloud relationship.
(3) Stage an unannounced vulnerability test on your cloud provider. Similar to when TSA is testing airport security with their security tiger team, you can execute a mock infiltration test scenario without the cloud provider security team knowing its coming. This will help you establish the confidence and TRUST that your data and systems have the highest level of protection (unless, they fail the test…)
(4) Walk before you run. Try before you buy. Date before your marry. Don’t put the most mission critical systems or applications in the cloud until you have confidence, cultural acceptance, and level of comfort through a cloud pilot. Build the TRUST needed through incremental successes and then go for the big enchilada when you’ve reached an appropriate milestone-- i.e cloud tested, verified, and trusted.
(5) Ask for weekly or monthly performance reports that show how effective and secure the cloud provider is performing. Make them earn your TRUST through metrics , quantitative analysis, and statistics.
(6) Tour the facilities where your cloud provider will host your solution. Make sure they show you exactly where your space will be located. You need to see your hosting space in person to TRUST that will be adequate.
(7) Ask for reference qualifications. See if others TRUST the provider as well. Ask the references how they overcame their trust concerns (peer advice).
(8) Ensure there are Key Performance Indicators that are contractually tied to the performance. This includes incentives and disincentives on their Service Level Agreements. Make them perform to the level of TRUST they have committed to.
(9) Keep a contingency plan and contingency architecture in case things don’t work out in the cloud and you need to drop back to Earth. If the TRUST fails and the cloud fails, you need a back-up plan so you are not left without the infrastructure to deliver your business capabilities. Cloud is a new, so a contingency is very important.
(10) Consult with diverse set of stakeholders in your organization to get their "cloud advice". Talk with the SOA, EA, IT Architects, developers, business users to get their opinions and recommendations. Make sure a broad group is surveyed to ensure cloud decisions are not made in a vac cum.

SOA Manifesto

Since I'm presenting on Agile SOA tomorrow at the MITRE conference, I thought it appropiate to create a slide on a proposed SOA Manifesto. Let me know your feedback:

1) Architecture driven (over product driven implementation)
2) Integrated Systems, Processes, and People (over individual silo's)
3) Business Visibility and Understanding (over IT priorities)
4) Re-usability (over redundant efforts)
5) Standards and Governance (over non structure)
6) Agility and Adaptability (over non flexibility)

Boiling SOA to 3 Simple Patterns

I love SOA. There I said it! It's done more than pay my bills, it's helped me strategize robust solutions to help businesses succeed. I can't see how any organizaiton can survive without a SOA an strategy! Ok, enough with the SOA love fest. Recently, I have been discussing and debating with some of my colleagues my thought process that Services should be the foundation for anything integration. Even when Ms. Maine proclaimed “SOA is Dead”, she followed up with “long live Services”. Let me explain my rationale. I see services as used and re-used in 3 simple, high-level integration patterns and this is how services can be the foundation for anything integration:

(1) System-to-System integration. Everyone should be very familiar with this pattern. This is 90% of all integration issues. Information Sharing. Data Exchanging. Synchronizing. Hetergenous systems, businesses, applications. Getting data from point A to point B in the right format, at the right time, and with the uptmost quality. This can be real-time or batch. It can be atomic transactions or bulk data loads. All of this should be used through services! Does this mean webServices? Not necessarily, as we know for example that ETL can’t afford the overhead of HTTP and XML. Not to beat a dead horse for the SOA experts, but I think many out there still don't understand that a service isn’t necessarily a webService. What is more important than webServices is the notion of architecting and exposing Business Services that can be consumed and re-used across enterprise. Not rocket science, and most SOA-educated get this!

(2) Business Process Management. A business process should ultimately simply be the orchestration of services across an end-to-end business process during runtime execution of the process. A BPM/BPEL should map business services to each Level 1 activity in a multi-level process flow or process map. Pretty simple: each activity is a business service at Level 1. This approach enables your business process, at runtime, to reuse the same services used in system-to-system integration from Pattern #1 and be easily managed and monitored. Ultimately, you can see where I am going-- our goal is to create a single version of the truth that can be used across all 3 patterns.

(3) Composite Applications: I recommend to clients to re-think their approach to building applications and that not everything requires development or integration to the back-end or legacy systems. Instead, they can focus on developing the business rules and the front-end functionality of these user-enabled applications. No longer do they require using "legacy" techniques that create complexity and more architecture hairballs; rather, they can leverage services to simplify all the plumbing required in a typical application development lifecycle. Why build 1-off websites, dashboards, reports, or mobile applications and re-invent the wheel with all the hard work required for interfacing, security, transformation, translation without re-using services that could already exist? Your creating a bigger architecture hairball! Instead, use a “mashup approach”, and re-use the same services from pattern #1 and #2 and make your life simpler! Re-usability! Hide the complexity and consume what has been exposed! See, my approach with SOA is to untangle the hairball that exists in organizations and not create more lines of code, more interfaces, more maintenance. If we are building another interface another integration to maintain for web or portal development, we are making the hairball worse. Instead, we should be using services to mask the complexity of integrating user-facing applications to legacy systems. This will allow us to reduce and wither away the hairball.

Architect Obstacles

I've been invited to participate in a podcast next week, to discuss challenges today's IT Architects face. I would like to open up to a broader audience, please send me any of your most pressing obstacles, and I will include it in our discussions.

I've listed a few that I have experienced most recently:

• Non-Enterprise IT Dept:Individual pockets exist within the IT Dept. Even though architects exist in various areas throughout the organization, it is a disparate, federated architecture without a centralized framework. IT groups are not thinking “big picture” on how their individual kingdoms impact the greater enterprise. They are architecting in silos.
• Businesses making IT decisions. This really annoys most IT departments and for good reason-- businesses don't know technology, IT feels undermined, but the business has the money! So, everyone says you "must sell to the business", but I say "don't leave IT behind!" We have businesses buying software, consulting, and trying to do architecture without IT's input. Often times, the Line of Business decisions doesn't necessarily fit into the architecture. This is why you find organizations with 3 BPM tools, 15 SOA products, and God-knows how many web and database products. All the vendors are licking their chops to sell to business, but remember that IT has the expertise to help with standards, architecture, and to bring together enterprise-level thought processes.
• Industry and Technical Standards overload. It's funny, because some IT areas have too many standards to learn and try to digest (webServices, Security), while others have too few (i.e. Cloud Computing). Architects are confused which standards to use and when, which to ignore, and how to incorporate and apply the selected standards across their enterprise.
• Architecture not tied to a “business initiative”. Because architecture often doesn't get tied to a specific business initiative, it often gets underfunded. Many times its a foundational IT initiative, and therefore businesses believe it doesn't help improve business (increase revenues)...This frustration is often a failure of the architecture team itself-- they have failed to show a direct linkage to benefits, and now can't prove their value. To their defense, that's not always easy task, but it must get done to validate their initiatives and ultimately prove their existence. Architecture is important, necessary, and should not be ignored-- but can quickly be undervalued or under appreciated by the rest of the organization.
• Educating stakeholders This is one of the most frustrating aspects of the Architects job. They need to teach others the importance of Enterprise Architecture, SOA, Cloud, etc. If you think architects are challenged educating their Line of Business on what architecture is, but guess what-- not everyone in IT “get it” either. That is why you always see the architects so stressed and pulling their hair out :-)
• Commoditization Many folks across the organization just assume architecture gets done and most don't even know it gets done. It's like when you move into a house, many of us don't appreciate all the work that went into building the house, all they way from digging the dirt to framing the house-- we only lavish at the beautiful bathrooms and granite counter tops. We don't see all the labor that occurred underneath. This is the same for users of software-- we only appreciate the end product and if the web page or dashboard looks nice. Certain things are expected to be there and to always work-- like a commodity. Therefore architecture team receives minimal respect and little patience from stakeholders and customers. Rarely does anyone know about all the blood, sweat, and tears that went into building and architecting the system from scratch!