Departing From Common Sense

A local government agency that deals with a lot of data wanted to use the services of a hometown entrepreneur ( a guy I am acquainted with) who "does Big Data" to help them figure out what Big Data could do in concert with their traditional Business Intelligence platform. I am sure there will be a way to fit a solution to a problem that does not completely exist. Reading this Forbes article about the key themes in the Big Data space was interesting to me for a couple of highlights cited :

Banana production in Central America is twice the rate of trash production in New York City - as an example of the hidden nuggets of wisdom you may glean from crunching through a ginormous amount of data. Being that there is a market for everything, the thought is it may be important for someone to know the precise relationship between banana production in Central America and trash production in New York City. 

That is the holy grail of big data analytics - to uncover stuff like this. The there is the next level of detail - how to keep the flow of such nuggets coming in at a steady clip and how to derive competitive advantage from them. There are any number of product companies out there who promise to do one or both. But the quantitative details on their case studies are sparse - an irony considering all they deal with is data and way too much of it.

The second relates to the obsolescence and irrelevance of KPIs in the brave new world of Big Data

..old-fashioned Business Intelligence (BI) with its insistence on pre-defined Key Performance Indicators (KPI) is no match to big data analytics, which does not require pre-defined “schema.” 

Not sure what KPIs have to do with having a pre-defined schema(or not)

..big data delivers “a command center that shows you what’s happening, not a dashboard with 40 KPI.”

It would seem commonsenical that when the deluge of data and factoids is upon us and our critical analysis abilities as humans is not skyrocketing to keep up, that we may need some help in the form of structure. Those old fashioned KPIs may be what it takes to introduce method to the madness.