Cloud Politics and the Role of Data Integration

A few weeks ago I wrote a post called, “Avoiding a Cloud Data Disaster.” The post made the following recommendations to organizations investing in software as a service (SaaS) applications and platforms like Salesforce CRM and Force.com:

1. Don’t Wait to Integrate
2. Make Data Quality a Top Priority
3. Get IT Involved Early
4. Think Big, Start Small
5. Beware of Point Solutions

Recently I’ve been involved in a number of discussions with people in IT organizations where #3 was clearly not followed, which typically means that #1 was ignored as well. There’s been a lot written about why SaaS applications are well suited and appealing to the line of business (easy to use, rapid deployment, capex vs. opex, etc.). But how is IT’s role evolving as cloud-based applications continue to proliferate? One IT director put it this way, “our job is to make sure that the business doesn’t hurt themselves!”

According to IDC’s survey research, security, availability, and performance are the primary barriers to success in the cloud. But from my experience, internal politics can often be a much bigger hurdle to get over. I  find that there are generally three common scenarios when it comes to the IT dynamic, roles, and responsibilities:

• The Edge of the Enterprise: Okay, I didn’t coin the term, but I like it and I think it’s appropriate for cloud computing in 2010.  SaaS application adoption in larger organizations often begins with an under-served department or business unit (also known as data silos!). Line of business (LOB) then selects a point application that is unsupported by IT and attempts to hand-code basic integration. At some point IT will need to get involved to find a way to stitch it all together.  I spoke to a new Salesforce.com customer recently that is planning to roll out 1000 seats with no IT involvement whatsoever! In this case, the IT department is too busy trying to figure out Oracle (con)Fusion. This is when LOB often “goes rogue” (Sarah Palin style I guess). The fireworks on calls that I’ve been on involving both sides are always interestig and the need for a comprehensive integration strategy that will deliver timely, relevant, and trustworthy data is often the forcing function that brings them together.
• The No-IT Approach: This is another reason for the phenomenal growth of cloud-based applications. SMB is where SaaS apps have their roots. But this is where #1 and #4 are critical. The sooner integration gets on the radar of the SaaS application implementation, whether there are IT resources or not, the better. Look for a mulitenant integration as a service solution that won’t hit the ceiling as your data volumes, sources, and complexity grow over time. Limited IT resources doesn’t mean that you have to sacrifice scalability and performance – two of the cornerstones of enterprise-class data integration solutions.
• The LOB Self-Service / IT Trust and Control Approach: Okay, I know that this may be the Holy Grail for most organizations, but the good news is that integration delivered as a true on-demand service can actually play a critical role in making this dream a reality. What if IT and LOB cloud actually collaborate to create a trusted source of integrated data that is controlled and managed by IT, but accessible by non-technical people who either don’t have time to wait for their requirements to be met or they don’t have IT resources in the first place. Recently David Linthicum and I did a podcast on the topic based on this post:  Staging or Direct – SaaS Integration Best Practices. He then wrote his own post on the topic where he made the following point:

“The use of intermediary staging where the data can be viewed, manipulated, and cleaned, insures that the data quality and any data governance required occurs consistently.”

The CIO post, “10 Big Cloud Trends for 2010” didn’t call out politics as a big trend, but they did concede that:

“Politics will drive decisions:  Cloud-based services have demystified compute services and increasingly business executives will ask IT managers about the cloud.”

So what are the cloud politics in your organization? Do you see a role for data integration to help bridge the IT and LOB divide as it relates to self-service, governance, trust, and control? Am I overstating the problem? Got any best practices and/or anecdotes to share?

Staging Data: The Right Approach for Cloud Integration?

Last week I wrote a post about the differences between direct and staging of data from SaaS/cloud applications. I received a very thorough comment on the Informatica Perspectives blog about the benefits of staging data that I think is worth summarizing here:

The main reasons I opt for the staging are:

• It enables better business control before the data is pushed from one system to the other. E.g. in SFDC you can have a prospect that you want to become a customer in SAP, you may need to control the data (match existing customer) or enrich it before you push it to SAP. The staging becomes a fire wall so no corrupted data in propagated into your information systems.
• It enables tracking and reconciliation of a business process. The staging area can also be used as a logging area, each time a data is manipulated, it is logged enabling the audit of any action and the visibility on any reconciliation process.
• It enables the addition of new sources or targets with reuse instead of building the spaghetti plate of point to point direct interfaces. It responds to the SOA paradigm.
• It breaks the dependencies between the two systems enabling asynchronous synchronization or synchronous with different size of data set (single message or bulk). And if one or the other system is down or in maintenance for a period of time, it does not affect the synchronization process because the data can be replayed (or compensated) from the staging area.

You can read the full comment here. Any other experiences or best practices to share? Do you agree / disagree? And what if you don’t have the IT resources for this type of architecture? Is direct for SMB and short-term tactical requirements only?