I guess everyone is sick and tired of seeing the term “big data” attached to just about everything larger than 1 kilobyte.
But who is responsible? Who do we hold accountable for overusing the term big data? Who killed big data?
Was it first and foremost the vendors who made the kill? A recent blog post called “Big Data is Dead. What’s Next?” by John De Goes suggest that the vendors are to be blamed for stabbing big data from behind.
Could it be the analysts? I have, as mentioned in the post The Big MDM Trend, seen how Gartner (the analyst firm) have put big data forward in the shouting gallery in order to explain something already explained with other terms.
Big data has often been personalized by the data scientist. So maybe it was a Californian girl called Jill Dyché who caused an extinction of the data scientist and thereby big data. She wrote the blog post called Why I Wouldn’t Have Sex with a Data Scientist.
What do you think? Who killed big data?
Like most new ideas, big data is going through the new concept lifecycle. There are those who embrace it, those who pan it and those who don’t care. Within the data world, we have seen data quality, data governance and master data management transition the lifecycle wave. Beneath the surface of the unbridled enthusiasm for big data and those professing its demise lies a gem of an idea.
Our society survives on data and algorithms. Increasingly decisions are being made based on algorithms created from data. A consumer loan is mostly predicated on a FICO score. This score is based on the analysis of lots of data. Medical care decisions are based on statistical probabilities obtained from data. The newest medical procedures consist of 68,000 codes up from approximately 10,000. Intelligent devices such as cars and increasingly appliances generate large volumes of data. The concept of The Internet of Things where more and more devices will be generating data will contribute to big data. So we are in a big data world. But our tools and capabilities to manage this data both technologically and sociologically are missing. It’s not that we need data scientists but that we need techniques to manage data scientifically.
I suggest looking beyond the surface hype, positive and negative and examine how data is evolving. The concept of big data has been evolving. The effects are subtle and profound. It’s already here but how to manage it and exploit it requires thought and experimentation and few of those writing about it have explored its depths. The big data concept is alive and well. The hype however maybe waning.
Richard, I agree. Big data is real, but there is just so much unreal talk about big data going around which may kill the real big data talk. And we have seen that before with other inflated expectations. This is just so big.
Size doesn’t matter! Data management best practices can be scaled up.
Besides, we should always be aware of one basic truth about data big or not so big: correlation does not necessarily imply causation. Just because we read about big data and how important it is, doesn’t imply it is important (that is perhaps except to the vendors and purveyors of big data solutions).
I don’t know if the talking about big data is unreal but most of what I read is unsubstantiated. It’s mostly marketing and sales jargon and should be considered from that viewpoint. Some of it is even entertaining 🙂 e.g. having sex with Data (the character from Star Trek)
When I first heard the term “Big Data” I was quite excited. Then I found out about volume, velocity and variety and was crushingly disappointed.
Big Data is what business people call it and therefore I think IT has chosen to focus on the wrong Vs. Today’s “Big Data” is tomorrow’s Business As Usual. I think people should focus on the following:-
Vision – What is it you want to do with data, what are your requirements?
Value – If your vision pans out what is the ROI/ROTI?
Veracity – Given data quality issues, how can you trust what the data is telling you.
Vocalism – How are you going to sort out signal to noise?
Thanks for adding in Dave. I like your thoughts and the saying: “Big Data” is tomorrow’s Business As Usual.
Big Data is a term used to quickly define the whole concept of mining unstructured data for “information”; its simply the next logical extension of yesterday’s data mining. Big Data is not dead. Love the header of this article though – got my attention for a few seconds.
Thanks for commenting Sri. Nice blog you got going by the way.