MDM

Avoid Duplicates by Avoiding Peer-to-Peer Integrations

Henrik Gabs LiliendahlLeave a comment

When working in Master Data Management (MDM) programs some of the main pain points always on the list are duplicates. As explained in the post Golden Records in Multi-Domain MDM this may be duplicates in party master data (customer, supplier and other roles) as well as duplicates in product master data, assets, locations and more.

Most of the data quality technology available to solve these problems revolves around identifying duplicates.  This is a very intriguing discipline where I have spent some of my best years. However, this is only a remedy to the symptoms of the problem and not a mean to eliminate the root cause as touched in the post The Good, Better and Best Way of Avoiding Duplicates.

The root causes are plentiful and as all challenges they involve technology, processes and people.

Having an IT landscape with multiple applications where master data are a created, updated and consumed is a basic problem and a remedy to that is the main reason of being for Master Data Management (MDM) solutions. The challenge is to implement MDM technology in a way that the MDM solution will not just become another silo of master data but instead be solution for sharing master data within the enterprise – and ultimately in the digital ecosystem around the enterprise.

blind-spot-take-careThe main enemy from a technology perspective is in my experience peer-to-peer system integration solutions. If you have chosen application X to support a business objective and application Y to support another business objective and you learn that there is an integration solution between X and Y available, this is very bad news. Because short term cost and timing considerations will make that option obvious. But in the long run it will cost you dearly if the master data involved are handled in other applications as well. Because then you will have blind spots all over the place where through duplicates will enter.

The only sustainable solution is to build a master data hub where through master data are integrated and thus shared with all applications inside the enterprise and around the enterprise. This hub must encompass a shared master data model and related metadata.

 

MDM Alternative Facts

Henrik Gabs LiliendahlLeave a comment

When searching for information about Master Data Management (MDM) solutions you will stumble on a lot of alternative facts.

Here are three more or less grave examples:

The MDM news is filled with yet a new market research report at sale for a few thousand US dollars. These reports look at first hand to be very thorough and information rich. But usually with a closer look you will become suspicious. It may be the mention of key players where often some are missing and a few actually mentioned will be companies more known from other trades. And the structure and content, as in the below example, seems to be a copy paste from other trades. Hmmm… “Production”, “Gross Margin” …. Seems to be more about the global cement market.

Market Research MDM.png

The next example is from an article called The 4 Best Master Data Management (MDM) Software Tools to Consider. Oracle, Profisee, Talend and SAP are all viable solutions. But Oracle seems to be going away from this market and the below justification for Oracle is very little about MDM.

Oracle MDM

Finally, on the pedantic side, even the recognized analyst firms can make a mistake (or a copy paste from earlier years). Forrester places Informatica as a German company. Well, it is the Product Information Management (PIM) wave and Informatica got into PIM (now Product 360 MDM) by buying the German PIM vendor Heiler in 2012.

Infa as German.pngNope, there is no such thing as a single version of the truth.

Learn from MDM Vendors

Henrik Gabs LiliendahlLeave a comment

The Disruptive List of Master Data Management Solutions has a blog. On this blog some of the leading MDM vendors provides guest blog posts with their perspective on both emerging trends and good old fundamental prerequisites within the MDM discipline.

Earlier this year David Corrigan of AllSight wrote a thought provoking post about 3 Reasons MDM No Longer Delivers a Customer 360.    

Yesterday Shamanth Shankar of Riversand contributed with a timely post on Why next generation MDM and PIM solutions must be in the Cloud.

Today we have a well-founded post by Nils Erik Pedersen of Stibo Systems on the Five Steps to Guarantee a Successful Master Data Management Implementation.

Learn from MDM Vendors

The Three MDM Ages

Henrik Gabs LiliendahlLeave a comment

Master Data Management (MDM) is relatively new discipline. The future will prove what is was, but standing here in mid-2018 I see that we already had 2 ages and are now slowly proceeding into a 3rd age. These ages can be coined as:

  • Pre MDM,
  • Middle MDM and
  • High MDM

Pre MDM

In these dark ages the term Master Data Management may have been used, but there were not any established discipline, methodologies, frameworks and technology solutions around that truly could count as MDM.

We had Customer Data Integration (CDI) around, we had Product Information Management (PIM) in the making and some of us were talking Data Quality Management – and that in practice being namely deduplication / data matching.

Middle MDM

MDM as Three Letter Acronym (TLA) emerged in the mid 00’s as told in the post Happy 10 Years Birthday MDM Solutions.

It was at that time Aaron Zornes changed his stage name from The Customer Data Integration Institute to The MDM Institute.

During this age many MDM solutions slowly but steadily have developed into multi-domain MDM solutions as reported over at the Disruptive MDM List in the blog post called 4 Vendor Paths to Multidomain MDM covering the road travelled by 10 vendors on the MDM market.

Most MDM solutions in the Middle MDM Age have been deployed on-premise

High MDM

We are now cruising into the High MDM Age. First and foremost a lot more organizations are now implementing MDM. Many new deployments are cloud based. New ways are tried out like encompassing more than master data in the same platform.

The jury is of course still out about what will be some main trends of the High MDM Age. My money is placed on what Gartner, the analyst firm, calls Multienterprise MDM as elaborated in the post Ecosystem Wide MDM.

MDM Ages.png

Three Remarkable Observations about Reltio

Henrik Gabs Liliendahl3 Comments

The latest entry on The Disruptive Master Data Management Solutions List is Reltio. I have been following Reltio for more than 5 years and have had the chance to do some hands on lately.

In doing that, I think there are three observations that makes the Reltio Cloud solution a remarkable MDM offering.

More than Master Data

While the Reltio solution emphasizes on master data the platform can include the data that revolves around master data as well. That means you can bring transactions and big data streams to the platform and apply analytics, machine learning, artificial intelligence and those shiny new things in order to go from a purely analytical world for these disciplines to exploit these data and capabilities in the operational world.

The thinking behind this approach is that you can not get a 360-degree on customer, vendor and other party roles as well as 360-degree on products by only having a snapshot compound description of the entity in question. You also need the raw history, the relationships between entities and access to details for various use cases.

In fact, Reltio provides not just operational MDM, but through a module called Reltio IQ also brings continuously mastered data, correlated transactions into an Apache Spark environment for analytics and Machine Learning. This eliminates the traditional friction of synchronizing data models between MDM and analytical environments. It also allows for aggregated results to be synchronized back into the MDM profiles, by storing them as analytical attributes. These attributes are now available for use in operational context, such as marketing segmentation, sales recommendations, GDPR exposure and more.

Multiple Storing Capabilities

There is an ongoing debate in the MDM community these days about if you should use relational database technology or NoSQL technology or graph technology? Reltio utilizes all three of them for the purposes where each approach makes the most sense.

Reference data are handled as relational data. The entities are kept using a wide column store, which is a technique encompassing scalability known from pure column stores but with some of the structure known from relational databases. Finally, the relationships are handled using graph techniques, which has been a recurring subject on this blog.

Reltio calls this multi-model polyglot persistence, and they embrace the latest technologies from multiple clouds such as AWS and Google Cloud Platform (GCP) under the covers.

Survival of the Fit Enough

One thing that MDM solutions do is making a golden record from different systems of records where the same real-world entity is described in many ways and therefore are considered duplicate records. Identifying those records is hard enough. But then comes the task of merging the conflicting values together, so the most accurate values survive in the golden record.

Reltio does that very elegantly by actually not doing it. Survivorship rules can be set up based on all the needed parameters as recency, provenance and more and you may also allow more than one value to survive as touched in the post about the principle of Survival of the Fit Enough.

In Reltio there is no purge of the immediately not surviving values. The golden record is not stored physically. Instead Reltio keeps one (or even more than one) virtual golden record(s) by letting the original source records stay. Therefore, you can easily rollback or update the single view of the truth.

The Reltio platform allows survivorship rules to be customized in rulesets for an unlimited number of roles and personas. In effect supporting multiple personalized versions of the truth. In an operational MDM context this allows sales, marketing, compliance, and other teams to see the data values that they care about most, while collaborating continuously in what Reltio calls the Self-Learning Enterprise.

Going beyond operational MDM

 

Achieving Business Benefits from Multi-Domain MDM

Henrik Gabs LiliendahlLeave a comment

Multi-Side MDMThe title of this blog post is also the title of my presentation at a Master Data Management (MDM) event that will take place in Berlin the 18th and 19th October 2018.

Here, I will give my perspectives on:

Read more about this MDM event from ThinkLinkers here. Hope to see you in Berlin.

PS: You can watch a YouTube video with testimonials from a previous event here.

MDM in The Cloud, On-Premise or Both

Henrik Gabs LiliendahlLeave a comment

One of the forms of Master Data Management (MDM) is the rising cloud deployment model as touched in the Disruptive MDM List blog post about 8 Forms of Master Data Management.

If we look at the MDM solution vendors, they may in that sense be divided into three kinds:

  • Cloud only, which are vendors born in the cloud age and who are delivering their service in the cloud only. Reltio is an example of that kind of MDM vendor.
  • Cloud or on-premise, which are vendors that can deliver both in the cloud and on premise, but where it makes most sense that you as a customer chooses the one that fits you the best. An example is Semarchy.
  • Cloud and on-premise. Informatica is the example of an MDM vendor that embraces both deployment models (together with other data management disciplines) at the same time (called hybrid) as told in an article by Kristin Nicole of SiliconANGLE. The title goes like this: Balancing act: Informatica straddles on-prem needs with cloud data at Informatica World 2018

Cloud MDM

Even With 20 Entities MDM Can be Hard

Henrik Gabs LiliendahlLeave a comment

This week I attended the Master Data Management Summit Europe 2018 and Data Governance Conference Europe 2018 in London.

Among the recurring sessions year by year on this conference and the sister conferences around the world will be Aaron Zornes presenting the top MDM Vendors as he (that is the MDM Institute) sees it and the top System Integrators as well.

Managing an ongoing list of such entities can be hard and doing it in PowerPoint does not make the task easier as visualized in two different shots captured via Twitter as seen below around the Top 19 to 22 European MDM / DG System Integrators:

20 entities

Bigger picture available here.

Now, the variations between these two versions of the truth and the real world are (at least):

  • Red circles: Is number 17 (in alphabetical order) Deloitte – in Denmark – who bought Platon 5 years ago or is it KPMG.
  • Blue arrow and circles: Is SAP Professional Services in there or not – and if they are, there must be 21 Top 20 players with two number 11: Edifixio and Entity Group
  • Green arrow: Number 1 (in alphabetical order) Affecto has been bought by number 8 CGI during this year.

PS: Recently I started a disruptive list of MDM vendors maintained by the vendors themselves. Perhaps the analysts can be helped by a similar list for System Integrators?

Interenterprise Metadata Handling

Henrik Gabs LiliendahlLeave a comment

I am aware that the title of this blog post is a bit geeky.

However, the terms are important in data management and your organisations ability to prosper in a continuously data driven world.

The term interenterprise was part of the previous post on this blog. The post was called What is Interenterprise Data Sharing? In a comment on LinkedIn analyst Simon Walker of Gartner explained interenterprise data sharing this way: “Interenterprise data sharing = Organizations are increasingly required to provide data to, and receive data from, external trading partners (customers, suppliers, business partners and others).”

Metadata is data about data. Handling metadata is an important facet of data management including in data governance, data quality management and Master Data Management (MDM). When it comes to the new trends in data management as big data and handling data in data lakes, the importance of metadata management will in my eyes become even more obvious.

In a current venture (Product Data Lake) we are working on building in metadata management for business ecosystems, meaning that trading partners can share product information either using the same metadata or linking their different metadata.

Using international, national and industry standards for product information will be the perfect solution within business ecosystem sharing of metadata and indeed this is the preferred option we support. However, there are many competing standards for product information and they come in developing versions, so having everyone on the same page at the same time is quite utopic.

Add to that everyone do not speak English – and even not the same variant of English. Metadata originates and should exist in the languages that is used in trading partnerships.

In Product Data Lake we have started out with these principles:

  • Product attributes can be tagged with an attribute type telling about what standard (if any) in terms of product identification, product classification or product feature it adheres to. More about that in the post Connecting Product Information.
  • Attribute short and long descriptions can be represented in different languages.
  • Trading partners can link their product attributes and have visibility in the Product Data Lake of the standards and descriptions used in the different languages they exist.

pdl-how-2

 

Diversities in Civil Registration

Henrik Gabs LiliendahlLeave a comment

Citizen Registry

The way governments around the world has organized their Master Data Management (MDM) is quite different. When it comes to registering citizens, the practice varies a lot as described in the post Citizen Master Data Management.

I have lived most of my years in Denmark where our national ID is unique and used for everything by public agencies and also a lot by private companies. Some years ago I lived in the United Kingdom, where the public agencies (and my bank) had no clue about who I were, when I came, what I did and when I left.

Recently the World Economic Forum has circulated some videos on LinkedIn telling about how stuff is done differently around the world. The video below is about the Danish civil registry (which by the way is similar in other Scandinavian countries):

What do you think? Would this public MDM and data quality practice work in USA, UK, Germany or where else you live?