When did the first data quality issue occur? Wikipedia says in the data quality article section titled history that it began with the mainframe computer in the United States of America.
Fellow data quality blogger Steve Sarsfield made a blog post a few years ago called A Brief History of Data Quality where it is said “Believe it or not, the concept of data quality has been touted as important since the beginning of the relational database”.
However, a predominant sentiment in the data quality realm is that data quality is not about technology. It is about people. People are the sinners of data quality flaws and as the main part of the problem people should also be the overwhelming part, if not the only part, of the solution.
So I guess data quality challenges were introduced when people showed up in the real world. How and when that happened is a matter of discussion as discussed in the blog post Out of Africa.
As explained in the post Movable Types the invention of movable types in printing some hundreds of years ago (the most important invention since someone invented the wheel for the first time) made a big boost in knowledge sharing among people – and also a big boost in data and information quality issues.
But I think the saying “To err is human, but to really foul things up you need a computer” is valid. Consequently I also think you may need a computer to help with cleaning up the mess and to prevent the mess from happening again. End of (hi)story.
This ninth Data Quality World Tour blog post is about Greece, a favorite travel destination of mine and the place of origin of so many terms and thoughts in today’s civilization.
Super senior citizens
Today Greece has a problem with keeping records over citizens. A recent data profiling activity has exposed that over 9,000 Greeks receiving pensions are over 100 years old. It is assumed that relatives has missed reporting the death of these people and therefore are taking care of the continuing stream of euro’s. News link here.
I found those good advices for you, when going to Greece today:
Timeliness: When coming to dinner, arriving 30 minutes late is considered punctual.
Accuracy: Under no circumstances should you publicly question someone’s statements.
Uniqueness: Meetings are often interrupted. Several people may speak at the same time.
(We all have some Greek in us I guess).
Previous Data Quality World Tour blog posts:
As humans we like to know about simple facts. As with weather forecasts we like to know exactly what temperature it’s going to be, if the sun will be shining or it’s going to be rain and sometimes also about the wind speed and direction relating to a given place and time in the future.
Meteorologists have struggled for ages to tell us about that. A traditional weather forecast will tell us the best guess for these few key indicators.
Many people today, including me, don’t really rely on the weather to do our work. But we may plan when to work, how to get to work and what to do besides work depending on the weather forecast.
So I usually study the weather forecast. Lately I have noticed that the Danish Meteorological Institute has experimented with how to visualize to the common people that the weather forecast is a best guess. So for example instead of having single colored blue plies indicating how much rain to expect, they now have the choice to have blue piles in different light or darker blue colors indicating the risk (or chance if you like) of rain.
Better data quality? I think so. Less confusing? I think not. It could be rain anytime. But it probably won’t.
A recurring subject for me and many others is talking and writing about people, processes and technology including which one is most important, in what sequence they must be addressed and, which is my main concern, how they must be aligned.
As we practically always are referring to the three elements in the same order being people, processes and technology there is certainly an implicit sequence.
If we look at maturity models related to data quality we will recognize that order too.
In the low maturity levels people are the most important aspect and the subject that needs the first and most attention and people are the main enablers for starting moving up in levels.
Then in the middle levels processes are the main concerns as business process reengineering enables going up the levels.
At the top levels we see implemented technology as a main component in the description of being there.
An example of the growing role of technology is (not surprisingly of course) in the data governance maturity model from the data quality tool vendor DataFlux.
One thing is sure though: You can’t move your organization from the low level to the high level by buying a lot of technology.
It is an evolutionary journey where the technology part comes naturally step by step by taking over more and more of the either trivial or extremely complex work done by people and where technology becomes an increasingly integrated and automated part of the business processes.
Thinking about it: Data Quality has a lot of similarities with parenting.
Some equivalence that comes to my mind is:
- Parenting must be done by everyone who has children; you are not supposed to have an education in education before being parents. The same about data. You are not supposed be a data quality expert before working with data; some common sense will bring you a long way.
- Some parenting experts never had their own children. I have seen the same with data quality experts too.
- Many people are more knowledgeable about how other people should raise children than about raising their own children. Same same with data quality.
- While we internally in the family may have some noise when parenting we keep that internally and keep up appearances to the outside. I think everyone have seen the same with data quality.
- There may be different styles in parenting going from “because I said so” to talking about it. The same is true around data quality improvement efforts.
- We do see more and more regulatory around parenting like it in my country now is forbidden to slap your kids. I think it should be forbidden to slap your naughty data too.
Just arrived home from summer vacation I have been thinking a bit about how we consultants act at work. On our vacation we used local guides at some places. These guides were our consultants at places they know very well and we didn’t know at all. But I also noticed they had some habits which may be considered as common weak sides of practicing consultancy.
Francisco Caballero has lived all his long life in the beautiful town Ronda in Southern Spain. He shared his great knowledge about the town with us in his distinguished blend of English and Spanish spiced up with some Russian, German and probably also Dutch words. I think we understood the most though we did have some variances when we compared our perceptions afterwards.
Besides telling about the town and the history behind Señor Caballero also shared his views about politics. He told about problems with young people today and increasing crime. He remembered things were much better when Generalissimo Franco was in charge. He admitted though that today there is no “bandidos” in the mountains as in the old days, but as he put it: “Today all bandidos in Madrid”. I guess he was referring to recent governments.
Robert is fifth generation of British descent living in Gibraltar, the small English enclave around the marvelous rock on the Southern tip of Spain facing Africa cross the narrow strait. I remember the opening scene of the James Bond film The Living Daylights is a hazardous car ride down the rock. Robert took us in his taxi on the very same narrow roads, practicing pretty much the same style of driving while explaining that as we had to go off and on the car all the time at the different sights, there was really no point in using the safety belts.
Personal commercial agenda
Salam seemed to know everyone and everything in Tangier, the Moroccan city on the Northern tip of Africa on the other side of the Strait of Gibraltar. Salam offered us a guided tour where we would go everywhere we wanted and look at everything we fancied using any time as we pleased. Only when going around he strongly urged us to go to exactly that spice shop he knew and strongly recommended not sitting at that café we spotted but preceding to a much better one. As infidels we couldn’t of course go into a mosque, unless (of course) we gave some extra Euro.
Following up on my previous post on Man versus Computer I am actually most workday mornings reminded about how man sucks.
Most workday mornings I leave home in my car heading into the following traffic:
- A 4 lane motorway rolling in from southern Copenhagen, rest of Denmark, Germany and ultimately rest of Eurasia.
- A 5th lane coming in from a local area.
These 5 lanes then split into:
- 2 lanes heading for the Danish answer to Silicon Valley (called Ballerup)
- 3 lanes leading to downtown Copenhagen or the main fair (called Bella Center), airport, Sweden and rest of Scandinavia.
Of course you will expect some mingling here. What happens every morning is rather a complete stop in traffic and the cause is not the merge and splitting but humans being drivers as:
- Experienced local selfish drivers staying in the fastest lane until they suddenly want to switch lane according to their ongoing route.
- Unexperienced (in this area) foreign drivers coming up from crowded central Europe in search for tranquility deep into the Swedish forests having no clue about where to position in this intersection. The same goes for Swedes returning for the opposite reason.
- Everyone else having fun rejecting the switching from the selfish types and the foreign ones who should know better than passing in rush hours.
Some solutions to this problem might be:
- Change Management learning people better driving habits.
- Onboard computer in every car taking care of lane positioning. Should go smooth splitting 5 lanes into 2 + 3 lanes.
Now I am waiting for which solution that will be implemented first.