7 Requirements for Cloud Application and Infrastructure Monitoring

This blog is focused on cloud data and application integration, with a lot of posts about analytics along the way. This presentation introduces the key cloud monitoring requirements for delivering enterprise digital intelligence in the world of increased cloud adoption, microservices, containers, serverless architecture, and artificial intelligence. You can read the full article here.

You can also get an idea of what this looks like in the world of DevOps in this short video:

The Business Intelligence Market – What’s Old is New

As the data visualization, big data, Hadoop, Spark and self-service hype gives way to IoT, AI and Machine Learning, I dug up an old parody post on the business intelligence market circa 2007-2009 when cloud analytics was just a disruptive idea.

Here’s the post in it’s entirety (with apologies to Billy Joel). Lots of companies here that I had forgotten about. I’m sure Claudia Imhoff has their coffee mugs at the #BBBT headquarters in Boulder…


This week the very funny Bubble 2.0 video was removed from YouTube for copyright reasons. In honor of the video, which was based on Billy Joel’s We Didn’t Start the Fire, here’s my tribute to the business intelligence market. Note, I’m no poet and it’s Friday! Thanks to The OLAP Report for lots of great market materials. Anybody want to shoot a video?

“We Couldn’t Get the Answers”
HP Lune, EF Codd, Chris Date, Stonebrake
APL, Express, Forest and Trees.

Comshare, Pilot, Metaphor, watch out here comes some more:
OLAP, ROLAP, HOLAP, MOLAP now my head hurts.

Ingres, Sybase, DB2, Informix, Oracle, Tandem too
Relational, ODBMS, how about we call it DSS?

EIS, client server, bring on the Web and SQL Server
OLAP for the masses, gents? No, let’s call it business intelligence.

We couldn’t get the answers
The tools took too much learning
For simplicity we’re yearning
We couldn’t get the answers
IT couldn’t find them
But they kept on buying them

MetaCube, Lightship, Beacon, Pillar, Holos, I’m starting to weaken
OLAP Services, TM1, Pablo, Wired, and Crystal fun.

Zambonini, Liautaud, Michael Saylor still unknown
Codd again, Netz bros, Kimball, Inmon, then Arbor goes.

Ad hoc query, data mining, information I’m still not finding
Data warehouse, what the hell! Just don’t forget about ETL.

Balanced scorecards, GIS, analytic apps, extranets
BI portals, real-time, dashboards and visualization look fine.

We still couldn’t get the answers
The tools took too much learning
For simplicity we’re yearning
We couldn’t get the answers
IT couldn’t find them
But they kept on buying them

Panorama, Andyne, Pilot, WhiteLight what a find!
Maximal, Gentia, and Next Action Technologies.

Sterling, Platinum and IA eventually all go to CA
Showcase, SQRIBE all get imbibed and don’t forget OLAP@ Work.

Comshare, Brio, AlphaBlox, Crystal’s on the chopping block
Siebel goes to Oracle, would Hyperion be the next vendor culled?

BOBJ on a shopping spree, Acta, Cartesis, SRC
Gartner bandwagon of CPM leads Cognos to Adaytum.

We couldn’t get the answers
The tools took too much learning
For simplicity we’re yearning
We couldn’t get the answers
IT couldn’t find them
But they kept on buying them

SOA, MDM, running out of acronyms

Still searching for the one truth, data silos hard to remove
Consolidate, standardize, “trough of disillusionment” lies.

Competency centers, shelfware, disparate data everywhere
Pay and pay, day after day, there has to be a better way!

We couldn’t get the answers
The tools took too much learning
For simplicity we’re yearning
We couldn’t get the answers
IT couldn’t find them
But they kept on buying them

Oracle buys Hyperion, SAP is on the run
Outlooksoft and Pilot go, now they’ve joined Netweaver’s show.

Waldorf / Paris HQs unite, “business users” told to delight
IBM buys Cognos too, what’s next for the SAS and MicroStrategy crew?

Open source, on-demand, innovation from a brand new land
Infrastructure, integration, what happened to my business question?
Gartner quads, Forester Waves, so many analytic predictions made
On-premise platform wars, will the SaaS model even the score?

We couldn’t get the answers
Will this business challenge go on and on and on….

3 Trends that are Changing the World of Data

upside_tdwiIn my last post, I wrote about the new data integration requirements. In this post I wanted to share a few points made recently in a TDWI institute interview with SnapLogic founder and CEO Gaurav Dhillon when he was asked:

What are some of the most interesting trends you’re seeing in the BI, analytics, and data warehousing space?

There are three trends that I believe are fundamentally changing the world of data.

The first is the shift to the cloud. Rapid provisioning, ease of use, and cost are just a few of the drivers as data gravity continues to shift.

(Data gravity means that when a large data set is sitting in a large Hadoop or Splunk instance in an on-premises system, it doesn’t make sense to load all that data into the cloud to run analytics functions. Instead, one would ship the function to the data and return results. Being able to do this seamlessly can greatly simplify integration pipelines.)

Also driving this trend is the fact that cloud data warehousing and analytics have moved from rogue departmental use cases to enterprise deployments.

The second trend is the data lake and how to complement, extend — and in some cases replace — the traditional data warehouse with a reference architecture that is built to handle all new and future sources and enable more proactive and predictive analytics.

The third trend is the Internet of Things (IoT). It’s already happening today in some industries with data velocity, variety, and, of course, volume. What’s more important, though, is the kinds of analytics and insights that will become possible because of IoT sensors, wearables, and devices — once organizations figure out how to separate signal from noise through the right data management techniques.

He was also asked about the drivers for modern integration platforms, whether you call it iPaaS or HIP or something else:

  1. The end of the ETL and ESB cycle
  2. The need for speed
  3. Hybrid infrastructure

Be sure to check out the entire interview and share your feedback.

The New Data Integration Requirements

This week SnapLogic posted a presentation of the 10 Modern Data Integration Platform Requirements on the company’s blog. They are:

  1. Application integration is done primarily through REST & SOAP services
  2. Large-volume data integration is available to Hadoop-based data lakes or cloud-based data warehouses
  3. Integration has to support the continuum of data velocities starting from batch all the way to continuous streams
  4. Integration is event-based rather than clock-driven
  5. Integration is primarily document- centric
  6. Integration is hybrid and spans cloud-cloud and cloud-ground scenarios
  7. Integration itself has to be accessible through SOAP/REST APIs
  8. Integration is all about connectivity… connectivity connectivity
  9. Integration has to be elastic
  10. Integration has to be delivered as a service

I’ve embedded the presentation below.

Data Management Requirements for the Enterprise Data Lake

snaplogic_logo_accelerateSnapLogic published Eight Data Management Requirements for the Enterprise Data Lake. They are:

  1. Storage and Data Formats
  2. Ingest and Delivery
  3. Discovery and Preparation
  4. Transformation and Analytics
  5. Streaming
  6. Scheduling and Workflow
  7. Metadata and Governance
  8. Security

The post originally appeared on ITBusinessEdge as a slide show. The company also recently hosted a webinar on Democratizing the Data Lake with Constellation Research and published 2 whitepapers from Mark Madsen.


It’s Time to Rethink Integration

Last week SnapLogic posted the company’s mission on their blog, which is focused on accelerating how enterprise integration technology is delivered in the enterprise. The company is growing rapidly and hiring. Here is the their mission statement:

It’s time to rethink integration.
Integrating data and applications with legacy products is slowing down your business.
What you’re running now is no longer running fast enough for today’s business needs.
But what if you could eliminate the pain of integration and get to its promise?
To prioritize outcomes over process.
To enable your enterprise with a modern solution.

With SnapLogic you can.

SnapLogic brings all your data together, at incredible speeds and
with an ease never known before.
Data, applications, and APIs—from any source, anywhere.
Instead of coding, you merely drag and drop intelligent connectors in the cloud.
They fit together seamlessly.
In a snap.

You’ll connect faster and more efficiently, enabling you to become a more agile business.
Whether it’s maximizing investments in big data, or getting new products to market faster.

Whatever your enterprise strategy, SnapLogic gets you there easier and quicker.
And wasn’t that what integration was supposed to do in the first place?
Don’t let your legacy integration solution be your legacy.
It’s time to let go of those old ideas.
It’s time to operate at the speed of modern business.

You can learn more about SnapLogic on the company’s website.




2016 Data Predictions from Gaurav Dhillon

gaurav_dhillon_snaplogicGaurav Dhillon, the co-founder and CEO of SnapLogic, who also co-founded Informatica in the early ’90’s and was CEO of that company for 12 years, posted 4 data predictions and 1 market prediction this week on LinkedIn and the SnapLogic blog. His predictions are as follows:

  1. Data Eats the World and Integration Strategies Will Drive Digital Transformation
  2. Rising Data Lakes will Drown the Warehouse
  3. IoT Goes from Over-Hyped to Emerging Reality
  4. Multi-Cloud is the New Reality
  5. Many so-called Unicorns (private companies valued at over $1Billion) Will Turn Into Unicorpses

I shared some of my 2016 predictions in my previous post and summarized the most popular 2015 SnapLogic blog posts here.

Happy New Year!


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 7,148 other followers

Follow Me on Twitter

Sheffield View

Trying to keep up with Big Data Vendor Landscape

A Passion for Research

Focusing on CRM, cloud computing, ERP and enterprise software

SnapLogic Blog

Accelerate Your Integration. Accelerate Your Business.

Learning by Shipping

products, development, management...

Laurie McCabe's Blog

Perspectives on the SMB Technology Market

Andreessen Horowitz

Software Is Eating the World

%d bloggers like this: