Yesterday I had the pleasure of attending the Informatica MDM 360 and Data Governance Summit in London including being in a panel discussing best practices for your MDM 360 journey. The rise of Artificial Intelligence (AI) in Master Data Management (MDM) was a main theme at this event.
Informatica has a track record of innovating in new technologies in the data management space while also acquiring promising newcomers in order to fast track their market offering. So it is with AI and MDM at Informatica too. Informatica currently has two tracks:
- clAIre – the clairvoyant component in the Informatica portfolio that “using machine learning and other AI techniques leverages the industry-leading metadata capabilities of the Informatica Intelligent Data Platform to accelerate and automate core data management and governance processes”.
- Informatica Customer 360 Insights which is the new branding of the recent AllSight acquisition. You can learn about that over at The Disruptive Master Data Management Solutions List in the entry about Informatica Customer 360 Insights.
At the Informatica event the synergy between these two tracks was presented as the Intelligent 360 View. Naturally, marketing synergies are the first results of an acquisition. Later we will – hopefully – see actual synergies when the technologies are to be aligned, positioned and delivered to customers who want to be an intelligent enterprise of the future.
As any other IT enabled discipline Master Data Management (MDM) continuously undergo a transformation while adopting emerging technologies. In the following I will focus on five trends that seen today seems to be disruptive:
MDM in the Cloud
According to Gartner the share of cloud-based MDM deployment has increased from 19% in 2017 year to 24 % in 2018 and I am sure that number will increase again this year. But does it come as SaaS (Software as a Service), PaaS (Platform as a Service) or IaaS (Infrastructure as a Service)? And what about DaaS (Data as a Service). Learn more in the post MDM, Cloud, SaaS, PaaS, IaaS and DaaS.
Extended MDM Platforms
There is a tendency on the Master Data Management (MDM) market that solutions providers aim to deliver an extended MDM platform to underpin customer experience efforts. Such a platform will not only handle traditional master data, but also reference data, big data (as in data lakes) as well as linking to transactions. Learn more in the post Extended MDM Platforms.
AI and MDM
There is an interdependency between MDM and Artificial Intelligence (AI). AI and Machine Learning (ML) depends on data quality, that is sustained with MDM, as examined in the post Machine Learning, Artificial Intelligence and Data Quality. And you can use AI and ML to solve MDM issues as told in the post Six MDM, AI and ML Use Cases.
IoT and MDM
The scope of MDM will increase with the rise of Internet of Things (IoT) as reported in the post IoT and MDM. Probably we will see the highest maturity for that first in Industrial Internet of Things (IIoT), also referred to as Industry 4.0, as pondered in the post IIoT (or Industry 4.0) Will Mature Before IoT.
Ecosystem wide MDM
Doing Master Data Management (MDM) enterprise wide is hard enough. But it does not stop there. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus we will have a need for working on the same foundation around master data. Learn more in the post Multienterprise MDM.
One of the hottest trends in the Master Data Management (MDM) world today is how to exploit Artificial Intelligence (AI) and ignite that with Machine Learning (ML).
This aspiration is not new. It has been something that have been going on for years and you may argue about when computerized decision support and automation goes from being applying advanced algorithms to being AI. However, the AI and ML theme is getting traction today as part of digital transformation and whatever we call it, there are substantial business outcomes to pursue.
As told in the post Machine Learning, Artificial Intelligence and Data Quality perhaps all use cases for applying AI is dependent on data quality and MDM is playing a crucial role in sustaining data quality efforts.
Some of the use cases for AI and ML in the MDM realm I have come across over the years are:
Translating between taxonomies: As reported in the post Artificial Intelligence (AI) and Multienterprise MDM emerging technologies can help in translating between the taxonomies in use when digital transformation sets a new bar for utilizing master data in business ecosystems.
Transforming unstructured to structured: A lot of data is kept in an unstructured way and to in order to systematically exploit these data in AI supported business process we need make data more structured. AI and ML can help with that too.
Data quality issue prevention: Simple rules for checking integrity and validating data is good – but unfortunately not good enough for ensuring data quality. AI is a way to exploit statistical methods and complex relationships.
Categorizing data: Digital transformation, spiced up with increasing compliance requirements, has made data categorization a must and AI and ML can be an effective way to solve this task that usually is not possible for humans to cover across an enterprise.
Data matching: Establishing a link between multiple descriptions of the same real-world entity across an enterprise and out to third party reference data has always been a pain. AI and ML can help as examined in the post The Art in Data Matching.
Improving insight: The scope of MDM can be enlarged to Extended MDM Platforms where other data as transactions and big data is used to build a 360-degree of the master data entities. AI and ML is a prerequisite to do that.
The previous post on this blog was called Machine Learning, Artificial Intelligence and Data Quality. In here the it was examined how Artificial Intelligence (AI) is impacted by data quality and how data quality can impact AI.
Master Data Management (MDM) will play a crucial role in sustaining the needed data quality for AI and with the rise of digital transformation encompassing business ecosystems we will also see an increasing need for ecosystem wide MDM – also called multienterprise MDM.
Right now, I am working with a service called Product Data Lake where we strive to utilize AI including using Machine Learning (ML) to understand and map data standards and exchange formats used within product information exchange between trading partners.
The challenge in this area is that we have many different classification systems in play as told in the post Five Product Classification Standards. Besides the industry and cross sector standards we still have many homegrown standards as well.
Some of these standards (as eClass and ETIM) also covers standards for the attributes needed for a given product classification, but still, we have plenty of homegrown standards (at no standards) for attribute requirements as well.
Add to that the different preferences for exchange methods and we got a chaotic system where human intervention makes Sisyphus look like a lucky man. Therefore, we have great expectations about introducing machine learning and artificial intelligence in this space.
Next week, I will elaborate on the multienterprise MDM and artificial theme on the Master Data Management Summit Europe in London.
Using machine learning (ML) and then artificial intelligence (AI) to automate business processes is a hot topic and on the wish list at most organizations. However, many, including yours truly, warn that automating business processes based on data with data quality issues is a risky thing.
In my eyes we need to take a phased approach and double use ML and AI to ensure the right business outcomes from AI automated business processes. ML and AI can be used to rationalize data and overcome data quality issues as exemplified in the post The Art in Data Matching.
Instead of applying ML and AI using a dirty dataset at hand for a given business process, the right way will be to use ML and AI to understand and asses relevant datasets within the organization and then use thereon rationalized data to be understood my machines and used for sustainable automation of business processes.
Most of these rationalized data will be master data, where there is a movement to include ML and AI in Master Data Management solutions by forward looking vendors as examined in the post Artificial Intelligence (AI) and Master Data Management (MDM).
Leading up to the Nordic Midsummer I am pleased to join Informatica and their co-hosts Capgemini and CGI at two morning seminars on how successful organizations can leverage data to drive their digital transformation, the needed data strategy and the urge to have a 360-view of data relationships and interactions.
My presentations will be an independent view on the question: What are the latest and hottest trends within Master Data Management?
In this session, I will give the audience a quick walk-through visiting some in vogue topics as MDM in the cloud, MDM for big data, embracing Internet of Things (IoT) within MDM, business ecosystem wide MDM and the impact of Artificial Intelligence (AI) on MDM.
The events will take place, and you can register to be there, as follows:
Even though that Master Data Management (MDM) has been around as a discipline for about 15 years now, there is still a lot of road to be covered for many organizations and for the discipline as a whole.
Some of the topics I find to be the most promising visit points on this journey are cloud deployment of MDM solutions, inclusion of Artificial Intelligence (AI) in MDM and multienterprise MDM.
Cloud deployment of MDM has increased slowly but steadily over the recent years. Quite naturally the implementation of MDM in the cloud will follow the general adoption of cloud solutions deployed in each organization as master data is the glue between the data held in each application. Doing MDM in the cloud or not is, as with most things in life, not a simple question with a yes or no answer, as there are different deployment styles as examined in the post MDM, Cloud, SaaS, PaaS, IaaS and DaaS.
Inclusion of Artificial Intelligence (AI) and Machine Learning (ML) in the MDM discipline will, in my eyes, be one of the hottest topics in the years to come. MDM is not the easiest IT enabled discipline in which AI and ML can be applied. Handling master data has many manual processes today because it is highly interactive, and the needed day-to-day decisions requires much knowledge input. But we will get there step by step and we must start now as told in the post It is time to apply AI to MDM and PIM.
Multienterprise MDM is emerging as a necessity following the rise of digitalization. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus, we will have a need for working on the same foundation around master data. This theme was pondered in the post Share or be left out of business.
The intersection between Artificial Intelligence (AI) and Master Data Management (MDM) – and the associated discipline Product Information Management (PIM) – is an emerging topic.
A use case close to me
In my work at setting up a service called Product Data Lake the inclusion of AI has become an important topic. The aim of this service is to translate between the different taxonomies in use at trading partners for example when a manufacturer shares his product information with a merchant.
In some cases the manufacturer, the provider of product information, may use the same standard for product information as the merchant. This may be deep standards as eCl@ss and ETIM or pure product classification standards as UNSPSC. In this case we can apply deterministic matching of the classifications and the attributes (also called properties or features).
However, most often there are uncovered areas even when two trading partners share the same standard. And then again, the most frequent situation is that the two trading partners are using different standards.
In that case we initially will use human resources to do the linking. Our data governance framework for that includes upstream (manufacturer) responsibility, downstream (merchant) responsibility and our ambassador concept.
As always, applying too much human interaction is costly, time consuming and error prone. Therefore, we are very eagerly training our machines to be able to do this work in a cost-effective way, within a much shorter time frame and with a repeatable and consistent outcome to the benefit of the participating manufacturers, merchants and other enterprises involved in exchanging products and the related product information.
Learning from others
This week I participated in a workshop around exchanging experiences and proofing use cases for AI and MDM. The above-mentioned use case was one of several use cases examined here. And for sure, there is a basis for applying AI with substantial benefits for the enterprises who gets this. The workshop was arranged by Camelot Management Consultants within their Global Community for Artificial Intelligence in MDM.
The concept of doing Master Data Management (MDM) not only enterprise wide but ecosystem wide was examined in the post Ecosystem Wide MDM.
As mentioned, product master data is an obvious domain where business outcomes may occur first when stretching your digital transformation to encompass business ecosystems.
The figure below shows the core delegates in the ecosystem wide Product Information Management (PIM) landscape we support at Product Data Lake:
Your enterprise is in the centre. You may have or need an in-house PIM solution where you manipulate and make product information more competitive as elaborated in the post Using Internal and External Product Information to Win.
At Product Data Lake we collaborate with providers of Artificial Intelligence (AI) capabilities and similar technologies in order to improve data quality and analyse product information.
As shown in the top, there may be a relevant data pool with a consensus structure for your industry available, where you exchange some of product information with trading partners. At Product Data Lake we embrace that scenario with our reservoir concept.
Else, you will need to make partnerships with individual trading partners. At Product Data Lake we make that happen with a win-win approach. This means, that providers can push their product information in a uniform way with the structure and with the taxonomy they have. Receivers can pull the product information in a uniform way with the structure and with the taxonomy they have. This product data syndication concept is outlined in the post Sell more. Reduce costs.