The Disruptive MDM / PIM List is list of solutions in the Master Data Management (MDM), Product Information Management (PIM) and Data Quality Management (DQM) space.
The list presents both larger solutions that also is included by the analyst firms in their market reports and smaller solutions you do not hear so much about, but may be exactly the solution that addresses the specific challenges you have.
The latest entry on the list, Reifier, is one of the latter ones.
Matching data records and identifying duplicates in order to achieve a 360-degree view of customers and other master data entities is the most frequently mentioned data quality issue. Reifier is an artificial intelligence (AI) driven solution that tackles that problem.
Read more about Reifier here.
Every time there is a survey about what causes poor data quality the most ticked answer is human error. This is also the case in the Profisee 2019 State of Data Management Report where 58% of the respondents said that human error is among the most prevalent causes of poor data quality within their organization.
This topic was also examined some years ago in the post called The Internet of Things and the Fat-Finger Syndrome.
Even the Romans new this as Seneca the Younger said that “errare humanum est” which translates to “to err is human”. He also added “but to persist in error is diabolical”.
So, how can we not persist in having human errors in data then? Here are three main approaches:
- Better humans: There is a whip called Data Governance. In a data governance regime you define data policies and data standards. You build an organizational structure with a data governance council (or any better name), have data stewards and data custodians (or any better title). You set up a business glossary. And then you carry on with a data governance framework.
- Machines: Robotic Processing Automation (RPA) has, besides operational efficiency, the advantage of that machines, unlike humans, do not make mistakes when they are tired and bored.
- Data Sharing: Human errors typically occur when typing in data. However, most data are already typed in somewhere. Instead of retyping data, and thereby potentially introduce your misspelling or other mistake, you can connect to data that is already digitalized and validated. This is especially doable for master data as examined in the article about Master Data Share.
Yesterday I had the pleasure of attending the Informatica MDM 360 and Data Governance Summit in London including being in a panel discussing best practices for your MDM 360 journey. The rise of Artificial Intelligence (AI) in Master Data Management (MDM) was a main theme at this event.
Informatica has a track record of innovating in new technologies in the data management space while also acquiring promising newcomers in order to fast track their market offering. So it is with AI and MDM at Informatica too. Informatica currently has two tracks:
- clAIre – the clairvoyant component in the Informatica portfolio that “using machine learning and other AI techniques leverages the industry-leading metadata capabilities of the Informatica Intelligent Data Platform to accelerate and automate core data management and governance processes”.
- Informatica Customer 360 Insights which is the new branding of the recent AllSight acquisition. You can learn about that over at The Disruptive Master Data Management Solutions List in the entry about Informatica Customer 360 Insights.
At the Informatica event the synergy between these two tracks was presented as the Intelligent 360 View. Naturally, marketing synergies are the first results of an acquisition. Later we will – hopefully – see actual synergies when the technologies are to be aligned, positioned and delivered to customers who want to be an intelligent enterprise of the future.
As any other IT enabled discipline Master Data Management (MDM) continuously undergo a transformation while adopting emerging technologies. In the following I will focus on five trends that seen today seems to be disruptive:
MDM in the Cloud
According to Gartner the share of cloud-based MDM deployment has increased from 19% in 2017 year to 24 % in 2018 and I am sure that number will increase again this year. But does it come as SaaS (Software as a Service), PaaS (Platform as a Service) or IaaS (Infrastructure as a Service)? And what about DaaS (Data as a Service). Learn more in the post MDM, Cloud, SaaS, PaaS, IaaS and DaaS.
Extended MDM Platforms
There is a tendency on the Master Data Management (MDM) market that solutions providers aim to deliver an extended MDM platform to underpin customer experience efforts. Such a platform will not only handle traditional master data, but also reference data, big data (as in data lakes) as well as linking to transactions. Learn more in the post Extended MDM Platforms.
AI and MDM
There is an interdependency between MDM and Artificial Intelligence (AI). AI and Machine Learning (ML) depends on data quality, that is sustained with MDM, as examined in the post Machine Learning, Artificial Intelligence and Data Quality. And you can use AI and ML to solve MDM issues as told in the post Six MDM, AI and ML Use Cases.
IoT and MDM
The scope of MDM will increase with the rise of Internet of Things (IoT) as reported in the post IoT and MDM. Probably we will see the highest maturity for that first in Industrial Internet of Things (IIoT), also referred to as Industry 4.0, as pondered in the post IIoT (or Industry 4.0) Will Mature Before IoT.
Ecosystem wide MDM
Doing Master Data Management (MDM) enterprise wide is hard enough. But it does not stop there. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus we will have a need for working on the same foundation around master data. Learn more in the post Multienterprise MDM.
One of the hottest trends in the Master Data Management (MDM) world today is how to exploit Artificial Intelligence (AI) and ignite that with Machine Learning (ML).
This aspiration is not new. It has been something that have been going on for years and you may argue about when computerized decision support and automation goes from being applying advanced algorithms to being AI. However, the AI and ML theme is getting traction today as part of digital transformation and whatever we call it, there are substantial business outcomes to pursue.
As told in the post Machine Learning, Artificial Intelligence and Data Quality perhaps all use cases for applying AI is dependent on data quality and MDM is playing a crucial role in sustaining data quality efforts.
Some of the use cases for AI and ML in the MDM realm I have come across over the years are:
Translating between taxonomies: As reported in the post Artificial Intelligence (AI) and Multienterprise MDM emerging technologies can help in translating between the taxonomies in use when digital transformation sets a new bar for utilizing master data in business ecosystems.
Transforming unstructured to structured: A lot of data is kept in an unstructured way and to in order to systematically exploit these data in AI supported business process we need make data more structured. AI and ML can help with that too.
Data quality issue prevention: Simple rules for checking integrity and validating data is good – but unfortunately not good enough for ensuring data quality. AI is a way to exploit statistical methods and complex relationships.
Categorizing data: Digital transformation, spiced up with increasing compliance requirements, has made data categorization a must and AI and ML can be an effective way to solve this task that usually is not possible for humans to cover across an enterprise.
Data matching: Establishing a link between multiple descriptions of the same real-world entity across an enterprise and out to third party reference data has always been a pain. AI and ML can help as examined in the post The Art in Data Matching.
Improving insight: The scope of MDM can be enlarged to Extended MDM Platforms where other data as transactions and big data is used to build a 360-degree of the master data entities. AI and ML is a prerequisite to do that.
The previous post on this blog was called Machine Learning, Artificial Intelligence and Data Quality. In here the it was examined how Artificial Intelligence (AI) is impacted by data quality and how data quality can impact AI.
Master Data Management (MDM) will play a crucial role in sustaining the needed data quality for AI and with the rise of digital transformation encompassing business ecosystems we will also see an increasing need for ecosystem wide MDM – also called multienterprise MDM.
Right now, I am working with a service called Product Data Lake where we strive to utilize AI including using Machine Learning (ML) to understand and map data standards and exchange formats used within product information exchange between trading partners.
The challenge in this area is that we have many different classification systems in play as told in the post Five Product Classification Standards. Besides the industry and cross sector standards we still have many homegrown standards as well.
Some of these standards (as eClass and ETIM) also covers standards for the attributes needed for a given product classification, but still, we have plenty of homegrown standards (at no standards) for attribute requirements as well.
Add to that the different preferences for exchange methods and we got a chaotic system where human intervention makes Sisyphus look like a lucky man. Therefore, we have great expectations about introducing machine learning and artificial intelligence in this space.
Next week, I will elaborate on the multienterprise MDM and artificial theme on the Master Data Management Summit Europe in London.
Using machine learning (ML) and then artificial intelligence (AI) to automate business processes is a hot topic and on the wish list at most organizations. However, many, including yours truly, warn that automating business processes based on data with data quality issues is a risky thing.
In my eyes we need to take a phased approach and double use ML and AI to ensure the right business outcomes from AI automated business processes. ML and AI can be used to rationalize data and overcome data quality issues as exemplified in the post The Art in Data Matching.
Instead of applying ML and AI using a dirty dataset at hand for a given business process, the right way will be to use ML and AI to understand and asses relevant datasets within the organization and then use thereon rationalized data to be understood my machines and used for sustainable automation of business processes.
Most of these rationalized data will be master data, where there is a movement to include ML and AI in Master Data Management solutions by forward looking vendors as examined in the post Artificial Intelligence (AI) and Master Data Management (MDM).
Leading up to the Nordic Midsummer I am pleased to join Informatica and their co-hosts Capgemini and CGI at two morning seminars on how successful organizations can leverage data to drive their digital transformation, the needed data strategy and the urge to have a 360-view of data relationships and interactions.
My presentations will be an independent view on the question: What are the latest and hottest trends within Master Data Management?
In this session, I will give the audience a quick walk-through visiting some in vogue topics as MDM in the cloud, MDM for big data, embracing Internet of Things (IoT) within MDM, business ecosystem wide MDM and the impact of Artificial Intelligence (AI) on MDM.
The events will take place, and you can register to be there, as follows:
Even though that Master Data Management (MDM) has been around as a discipline for about 15 years now, there is still a lot of road to be covered for many organizations and for the discipline as a whole.
Some of the topics I find to be the most promising visit points on this journey are cloud deployment of MDM solutions, inclusion of Artificial Intelligence (AI) in MDM and multienterprise MDM.
Cloud deployment of MDM has increased slowly but steadily over the recent years. Quite naturally the implementation of MDM in the cloud will follow the general adoption of cloud solutions deployed in each organization as master data is the glue between the data held in each application. Doing MDM in the cloud or not is, as with most things in life, not a simple question with a yes or no answer, as there are different deployment styles as examined in the post MDM, Cloud, SaaS, PaaS, IaaS and DaaS.
Inclusion of Artificial Intelligence (AI) and Machine Learning (ML) in the MDM discipline will, in my eyes, be one of the hottest topics in the years to come. MDM is not the easiest IT enabled discipline in which AI and ML can be applied. Handling master data has many manual processes today because it is highly interactive, and the needed day-to-day decisions requires much knowledge input. But we will get there step by step and we must start now as told in the post It is time to apply AI to MDM and PIM.
Multienterprise MDM is emerging as a necessity following the rise of digitalization. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus, we will have a need for working on the same foundation around master data. This theme was pondered in the post Share or be left out of business.
The intersection between Artificial Intelligence (AI) and Master Data Management (MDM) – and the associated discipline Product Information Management (PIM) – is an emerging topic.
A use case close to me
In my work at setting up a service called Product Data Lake the inclusion of AI has become an important topic. The aim of this service is to translate between the different taxonomies in use at trading partners for example when a manufacturer shares his product information with a merchant.
In some cases the manufacturer, the provider of product information, may use the same standard for product information as the merchant. This may be deep standards as eCl@ss and ETIM or pure product classification standards as UNSPSC. In this case we can apply deterministic matching of the classifications and the attributes (also called properties or features).
However, most often there are uncovered areas even when two trading partners share the same standard. And then again, the most frequent situation is that the two trading partners are using different standards.
In that case we initially will use human resources to do the linking. Our data governance framework for that includes upstream (manufacturer) responsibility, downstream (merchant) responsibility and our ambassador concept.
As always, applying too much human interaction is costly, time consuming and error prone. Therefore, we are very eagerly training our machines to be able to do this work in a cost-effective way, within a much shorter time frame and with a repeatable and consistent outcome to the benefit of the participating manufacturers, merchants and other enterprises involved in exchanging products and the related product information.
Learning from others
This week I participated in a workshop around exchanging experiences and proofing use cases for AI and MDM. The above-mentioned use case was one of several use cases examined here. And for sure, there is a basis for applying AI with substantial benefits for the enterprises who gets this. The workshop was arranged by Camelot Management Consultants within their Global Community for Artificial Intelligence in MDM.