Cloud Integration: Batch vs. Real Time

In the last few days there have been diametrically opposite messages coming from two vendors who claim to deliver best-in-class data integration solutions for Salesforce.com (and other SaaS application vendors). Doug Henschen wrote about both cloud data integration product launches.

1. The first vendor focused on data migration to the cloud.  They acknowledged that data migration “often turns out to be more complex than expected” and highlighted the importance of performance to data integration (while throwing a few unsubstantiated haymakers at the competition I might add).
2. The second vendor focused on real-time application integration, making the claim that their product, “offers two-way, system-to-system, event-driven integration that is real-time rather than batch oriented.”

Hmmm. So which is more important? Real time or batch data integration? And is it really an either-or proposition? Are we really now dragging the old ETL vs. EAI debate to the cloud?

To me it gets back to a topic I’ve written about before: What to Look for in a Cloud Data Integration Solution.

If a data integration vendor claims to be all about one or the other it should be a red flag. The bottom line is to understand your requirements, your data volumes (both today and long term), your integration complexity (both today and long term), your resources (on-premise or SaaS?), and the skill set of your users (SaaS administrators and IT roles). And when it comes to a Salesforce.com partner, be sure to dig into customer references and read the reviews on the AppExchange. This will reveal quite about about overall customer adoption and success.

Feedback?

Staging Data: The Right Approach for Cloud Integration?

Last week I wrote a post about the differences between direct and staging of data from SaaS/cloud applications. I received a very thorough comment on the Informatica Perspectives blog about the benefits of staging data that I think is worth summarizing here:

The main reasons I opt for the staging are:

• It enables better business control before the data is pushed from one system to the other. E.g. in SFDC you can have a prospect that you want to become a customer in SAP, you may need to control the data (match existing customer) or enrich it before you push it to SAP. The staging becomes a fire wall so no corrupted data in propagated into your information systems.
• It enables tracking and reconciliation of a business process. The staging area can also be used as a logging area, each time a data is manipulated, it is logged enabling the audit of any action and the visibility on any reconciliation process.
• It enables the addition of new sources or targets with reuse instead of building the spaghetti plate of point to point direct interfaces. It responds to the SOA paradigm.
• It breaks the dependencies between the two systems enabling asynchronous synchronization or synchronous with different size of data set (single message or bulk). And if one or the other system is down or in maintenance for a period of time, it does not affect the synchronization process because the data can be replayed (or compensated) from the staging area.

You can read the full comment here. Any other experiences or best practices to share? Do you agree / disagree? And what if you don’t have the IT resources for this type of architecture? Is direct for SMB and short-term tactical requirements only?