Technology

Excellence vs Excel

Henrik Gabs Liliendahl2 Comments

We all use Excel though we know it is bad. It is a user friendly and powerful tool, but there are plenty of stories out there where Excel has caused so much trouble like this one from Computerworld in 2008 when the credit crunch struck.

I guess all people who works in data management curses Excel. Data kept in Excel is a pain  – you know where – as it is hard to share, you never know if you have the latest version, nice informative colouring disappears when transforming, narrow columns turns into rubbish, different formatting usually makes it practically impossible to combine two sheets and heaps of other not so nice behaviours.

Even so, Excel is still the most used tool for many crucial data management purposes as for example reported in the post The True Leader in Product MDM.

Excel is still a very frequent used option when it comes to exchanging data as touched by Monica McDonnell of Informatica in a recent blog post on Four Technology Approaches for IDMP Data Management.

Probably, the use of Excel as a mean to exchange data between organizations is the field where it will be most difficult to eliminate the dangerous use of Excel. The problem is that the alternative usually is far too rigid. The task of achieving consensus between many organizations on naming, formatting and all the other tedious stuff makes us turn to Excel.

Excellence vs Excel

When working with data quality within data management we may wrongly strive for perfection. We should rather strive for excellence, which is something better than the ordinary. In this case Excel is the ordinary. As Harriet Braiker said: “Striving for excellence motivates you; striving for perfection is demoralizing.”

In order to be excellent, though not perfect, in data sharing, we must develop solutions that are better than Excel without being too rigid. Right now, I am working on a solution for sharing product data being of that kind. The service is called the Product Data Lake.

The Future of Master Data Management

Henrik Gabs Liliendahl3 Comments

Back in 2011 Gartner, the analyst firm, predicted that these three things would shape the Master Data Management (MDM) market:

  • Multi-Domain MDM
  • MDM in the Cloud
  • MDM and Social Networks

The third point was in 2012, after the raise of big data, rephrased to MDM and Big Data as reported in the post called The Big MDM Trend.

In my experience all these three themes are still valid with slowly but steadily uptake.

open-doorBut, have any new trends showed up in the past years?

In a 2015 post called “Master Data Management Merger Tardis and The Future of MDM” Ramon Chen of Reltio puts forward some new possibilities to be discussed, among those Machine Learning & Cognitive computing. I agree with Ramon on this theme, though these have been topics around in general for decades without really breaking through. But we need more of this in MDM for sure.

My own favourite MDM trend is a shift from focussing on internally captured master data to collaboration with external business partners as explained in the post MDM 3.0 Musings.

In that quest, I am looking forward to my next speaking session, which will be in Helsinki, Finland on the 8th December. There is an interview on that with yours truly available on the Talentum Master Data Management 2015 site.

It is Magic Quadrant Week

Henrik Gabs Liliendahl2 Comments

Earlier this week this blog featured the Magic Quadrant for Customer MDM and the Magic Quadrant for Product MDM. Today it is time to have a look at the just published Magic Quadrant for Data Quality Tools.

Last year I wondered if we finally will see that data quality tools will focus on other pain points than duplicates in party data and postal address precision as discussed in the post The Multi-Domain Data Quality Tool Magic Quadrant 2014 is out.

Well, apparently there still isn’t a market for that as the Gartner report states: “Party data (that is, data about existing customers, prospective customers, citizens or patients) remains the top priority for most organizations: Almost nine in 10 (89%) of the reference customers surveyed for this Magic Quadrant consider it a priority, up from 86% in the previous year’s survey.”

Multi-Domain MDM and Data Quality DimensionsFrom own experience in working predominantly with product master data during the last couple of years there are issues and big pain points with product data. They are just different from the main pain points with party master data as examined in the post Multi-Domain MDM and Data Quality Dimensions.

I sincerely believes that there are opportunities in providing services to solve the specific data quality challenges for product master data, that, according to Gartner, “is one of the most important information assets an organization has; second-only, perhaps, to customer master data”. In all humbleness, my own venture is called the Product Data Lake.

Anyway, as ever, Informatica is our friend when it comes to free copies of a data management quadrant. Get a free copy of the 2015 Magic Quadrant for Data Quality Tools here.

Integration Matters

Henrik Gabs Liliendahl2 Comments

A recent report from KDR Recruitment takes a snapshot of the current state of the world of data in order to uncover some of the most pressing issues facing the Information Management industry and get a sense of what changes may be on the horizon.

One of the clearest findings was around what drives the selection of information software. The report states: “New software must integrate easily into existing infrastructure and systems. This is far and away the most important consideration for users, who also want that same flexibility to extend to customisation options and reporting functionalities.”

The graphic looks like this:

Integration

The ease of integration is in my experience indeed a very important feature when selecting (and selling) a data management tool. Optimally it should not be so because you can end up with not solving the business issue in a nice integrated way. But without integration a new data management tool will live in yet another silo probably only solving some part of the business issue.

The report from KDR Recruitment also covers where you use data to improve performance, the barriers to implementing an informational management strategy and other data management topics.You can read the full report called Not waving but drowning – The State of Data 2015 here.

PS: Kudos to KDR Recruitment for actually engaging in the sector where they work and doing so on social media. Very much in contrast to recruiters who just spam LinkedIn groups with their job openings.

Bookmark and Share

The Data Quality Market Just Passed 1 Billion USD

Henrik Gabs LiliendahlLeave a comment

The Data Quality Landscape – Q1 2015 from Information Difference is out. A bit ironically, the report states that the data quality market for the calendar year 2014 was worth a fraction over $1 billion. As the $ sign  could mean a lot of different currencies like CAD, AUD or FJD this statement is very ambiguous, but I guess Andy Hayler means USD.

dollarWhile there still is a market for standalone data quality tools an increasing part of data quality tooling is actually made with tools being a Master Data Management (MDM) tool, a Data Governance tool, an Extract Load and Transform (ETL) tool, a Customer Relationship Management (CRM) tool or an other kind of tool or software suite.

This topic was recently touched on this blog in the post called Informatica without Data Quality? Herein the reasons behind why the new owners of Informatica did not mention data quality as a future goodie in the Informatica toolbox was examined.

In a follow up mail an Informatica officer explained: “As you know Data Quality has become an integral part of multidomain MDM and of the MDM fueled Product Catalog App. We still serve pure DQ (Data Quality) use cases, but we see a lot growth in DQ as part of MDM initiatives”.

You can read the full DQ Landscape 2015 here.

Bookmark and Share

Informatica without Data Quality?

Henrik Gabs Liliendahl4 Comments

This week it was announced that Informatica, a large data management tool provider, will be taken over by a London based private equity firm and a Canadian pension invest management organization.

shark_eatThe first analyst reactions and line up of the potential benefits and the potential drawbacks can be found here on searchCIO in an article called Informatica going private could be a good thing for CIOs.

Most quotes in this article are from Ted Friedman, the Gartner analyst who writes the data quality tool magic quadrant, and Friedman notes, that the new owners doesn’t mention data quality as one of the goodies in the Informatica toolbox (opposite to data security, an area Informatica is not well known for).

So, maybe the new owners just don’t know yet what they bought, or they have a clear vision for the data management market where data quality is just being a natural part of cloud integration, master data management, data integration for next-generation analytics, and data security. The alternative routes could be decommissioning or split of, both familiar routes for this kind of take over.

Splitting of the data quality components should not be too hard, as some of these components has come to Informatica as acquisitions of Similarity Systems from Ireland and Identity Systems, which once was SSA with roots in Australia. I was actually a bit surprised when watching an Informatica presentation in London last autumn that the data quality part was the good old SSA Name3 service.

Bookmark and Share

No plan of operations extends with any certainty beyond the first contact with the full load of data

Henrik Gabs Liliendahl2 Comments

There is a famous saying from the military world stating that: “No plan survives contact with the enemy.” At least one blogger has used the paraphrasing saying: “No plan survives contact with the data.” A good read by the way.

Helmuth_Karl_Bernhard_von_Moltke
Helmuth von Moltke the Elder

Like most famous sayings also this phrase is simplified from the original version. The military observation made by Helmuth von Moltke the Elder is in full length: “No plan of operations extends with any certainty beyond the first contact with the main hostile force.”

Translating the extended military learning into data management makes a lot of sense too. You may plan data management activities using selected examples and you may test those using nice little samples. Like skirmishes before the real battle in warfare. But if your data management solution goes live on the full load of data for the first time, there most often is news for you.

From my data matching days I remember this clearly as explained in the post Seeing is Believing.

The mitigation is to test with a full load of data before going live. In data management we actually have a realistic way of overcoming the observation made by Field Marshall Helmuth Carl Bernard Graf von Moltke and revisit our plan of operations before the second and serious contact with the full load of data.

Bookmark and Share

Business Agility, Continuous Improvement and MDM

Henrik Gabs LiliendahlLeave a comment

Being able to react to market changes in an agile way is the path to the survival of your business today. As you may not nail it in the first go, the ability to correct with continuous improvement is the path for your business to stay alive.

open-doorDoing business process improvement most often involves master data as examined in the post Master Data and Business Processes. The people side of this is challenging. The technology side isn’t a walkover either.

When looking at Master Data Management (MDM) platforms in sales presentations it seems very easy to configure a new way of orchestrating a business process. You just drag and drop some states and transitions in a visual workflow manager. In reality, even when solely looking at the technical side, it is much more painful.

MDM solutions can be hard to maneuver. You have to consider existing data and the data models where the data sits. Master data is typically used with various interfaces across many business functions and business units. There are usually many system integrations running around the MDM component in an IT landscape.

A successful MDM implementation does not just cure some pain points in business processes. The solution must also be able to be maneuvered to support business agility and continuous improvement. Some of the data quality and data governance aspects of this is explored in the post Be Prepared.

Bookmark and Share

To-Be Business Rules and MDM

Henrik Gabs LiliendahlLeave a comment

checklistAn important part of implementing Master Data Management (MDM) is to capture the business rules that exists within the implementing organization and build those rules into the solution. In addition, and maybe even more important, is the quest of crafting new business rules that helps making master data being of more value to the implementing organization.

Examples of such new business rules that may come along with MDM implementations are:

  • In order to open a business account you must supply a valid Legal Entity Identifier (like Company Registration Number, VAT number or whatever applies to the business and geography in question)
  • A delivery address must be verified against an address directory (valid for the geography in question)
  • In order to bring a product into business there is a minimum requirement for completeness of product information.

Creating new business rules to be part of the to-be master data regime highlights the interdependency of people, process and technology. New technology can often be the driver for taking on board such new business rules. Building on the above examples such possibilities may be:

  • The ability to support real time pick and check of external identifiers
  • The ability to support real time auto completion and check of postal addresses
  • The ability to support complex completeness checks of a range of data elements

Bookmark and Share

Be Prepared

Henrik Gabs Liliendahl3 Comments

Working with data governance and data quality can be a very backward looking quest. It often revolves around how to avoid a recent data disaster or catching up with the organizational issues, the process orchestration and new technology implementations needed to support current business objectives with current data types in a better way.

This may be hard enough. But you must also be prepared for the future.

open-doorThe growth of available data to support your business is a challenge today. Your competitors take advantage of new data sources and better exploitation of known data sources while you are sleeping. New competitors emerge with business ideas based on new ways of using data.

The approach to inclusion of new data sources, data entities, data attributes and digital assets must be a part of your data governance framework and data quality capability. If you are not prepared for this, your current data quality will not only be challenged by decay of current data elements but also of not sufficiently governed new data elements or lack of business agility because you can’t include new data sources and elements in a safe way.

Some essentials in being prepared for inclusion of new kinds of data are:

  • A living business glossary that facilitates a shared understanding of new data elements within your organization including how they relate to or replaces current data elements.
  • Configurable data quality measurement facilities, data profiling functionality and data matching tools so on-boarding every new data element doesn’t require a new data quality project.
  • Self-service and automation being the norm for data capture and data consumption. Self-service must be governed both internally in your organization and externally as explained in the post Data Governance in the Self-Service Age.

Bookmark and Share