We all use Excel though we know it is bad. It is a user friendly and powerful tool, but there are plenty of stories out there where Excel has caused so much trouble like this one from Computerworld in 2008 when the credit crunch struck.
I guess all people who works in data management curses Excel. Data kept in Excel is a pain – you know where – as it is hard to share, you never know if you have the latest version, nice informative colouring disappears when transforming, narrow columns turns into rubbish, different formatting usually makes it practically impossible to combine two sheets and heaps of other not so nice behaviours.
Even so, Excel is still the most used tool for many crucial data management purposes as for example reported in the post The True Leader in Product MDM.
Excel is still a very frequent used option when it comes to exchanging data as touched by Monica McDonnell of Informatica in a recent blog post on Four Technology Approaches for IDMP Data Management.
Probably, the use of Excel as a mean to exchange data between organizations is the field where it will be most difficult to eliminate the dangerous use of Excel. The problem is that the alternative usually is far too rigid. The task of achieving consensus between many organizations on naming, formatting and all the other tedious stuff makes us turn to Excel.
When working with data quality within data management we may wrongly strive for perfection. We should rather strive for excellence, which is something better than the ordinary. In this case Excel is the ordinary. As Harriet Braiker said: “Striving for excellence motivates you; striving for perfection is demoralizing.”
In order to be excellent, though not perfect, in data sharing, we must develop solutions that are better than Excel without being too rigid. Right now, I am working on a solution for sharing product data being of that kind. The service is called the Product Data Lake.
Excel is abused for so many things because it is a tool that most people have available to them.
My experience is that asking for a tool outside of the approved corporate list of tools results in stunned silence quickly followed by almost total paralysis.
The amount of time and effort spent arguing for and justifying a tool to help measure data quality or process data intelligently can easily add a 5 figure sum to the price of the tool. This reduces the ROI to such an extent that the payback time becomes unattractive.
Then there is the prioritisation of the IT project list. I know of one set of data exchanges between companies that relied on Excel for well over 2 years simply because the timescales to build a proper data sharing capability kept falling off the bottom of the project priority list.
I have also seen Excel sheets and Macros that were an actual true reflection of what the business ACTUALLY did rather than what they told the business analysts they did. At the heart of the mechanical abomination was actually a simple and straight forward process that a small competent IT team could have produced with sufficient rigour to be considered production grade. Due to the make do and mend attitude towards data husbandry the physical implementation was the equivalent of a lever with a million moving parts.
Thanks for adding in Dave. The point about the corporate approved list of tools is close to me in an ironic way. We are working on MDM requirements for a huge list of possible attributes. We eagerly seek to eliminate the use of excel and instead use the MDM features. Due to lack of an approved feasible tool, we gather and share the requirements – in Excel sheets floating around.