The Road Ahead for MDM

Even though that Master Data Management (MDM) has been around as a discipline for about 15 years now, there is still a lot of road to be covered for many organizations and for the discipline as a whole.

vestre kirkegaardSome of the topics I find to be the most promising visit points on this journey are cloud deployment of MDM solutions, inclusion of Artificial Intelligence (AI) in MDM and multienterprise MDM.

Cloud deployment of MDM has increased slowly but steadily over the recent years. Quite naturally the implementation of MDM in the cloud will follow the general adoption of cloud solutions deployed in each organization as master data is the glue between the data held in each application. Doing MDM in the cloud or not is, as with most things in life, not a simple question with a yes or no answer, as there are different deployment styles as examined in the post MDM, Cloud, SaaS, PaaS, IaaS and DaaS.

Inclusion of Artificial Intelligence (AI) and Machine Learning (ML) in the MDM discipline will, in my eyes, be one of the hottest topics in the years to come. MDM is not the easiest IT enabled discipline in which AI and ML can be applied. Handling master data has many manual processes today because it is highly interactive, and the needed day-to-day decisions requires much knowledge input. But we will get there step by step and we must start now as told in the post It is time to apply AI to MDM and PIM.

Multienterprise MDM is emerging as a necessity following the rise of digitalization. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus, we will have a need for working on the same foundation around master data. This theme was pondered in the post Share or be left out of business.

MDM, Cloud, SaaS, PaaS, IaaS and DaaS

A while ago the trend of having the possibility to deploy a Master Data Management (MDM) solution in the cloud was covered in the post The Rise of Cloud MDM.

The latest Gartner MDM Magic Quadrant report has some numbers on that trend as mentioned in the post Who Will Make the Next Disruption on the MDM Market? Cloud based deployment has increased from 19% in 2017 year to 24 % in 2018 among Gartner’s respondents. While the organizations included here are the larger ones, I will guestimate that the cloud portion of MDM implementations are higher among midsize and smaller organizations.

As mentioned in the Gartner report there are however some confusion about what a cloud MDM solution really is. Does it come as SaaS (Software as a Service), PaaS (Platform as a Service) or IaaS (Infrastructure as a Service)? In this spectrum the vendor will provide most things in a SaaS solution, lesser stuff as PaaS and only the ability for the software to be hosted somewhere out there as IaaS.

One “as a Service” component in relation to master data you could expect in SaaS, but not necessarily in IaaS, is DaaS (Data as a Service) as for example out-of-the-box address verification and business directory integration services. A common address verification service is the one from Loqate, while Informatica though have their own solution based on their AddressDoctor acquisition. The most common business directory provider is Dun & Bradstreet.

Else the difference follows the general difference between SaaS, PaaS and IaaS which is about what the organization has do themselves (or through system integrators) around software updates, configuration, maintenance, monitoring and more.

On the brink to 2019 my guess is that we will see more MDM in the cloud next year as well as a movement from IaaS over PaaS to SaaS. This will include more DaaS covering more master data domains not at least in the product data space – a reason of being for the Product Data Lake service I am involved with.

Cloud MDM

The Rise of Cloud MDM

Cloud as a deployment method for Master Data Management (MDM) solutions is on the rise.

In the latest MDM vendor selection activities I am involved in cloud is not an absolute must but certainly the preferred deployment method.

The MDM vendor market is responding to that trend. Some of the new players offers purely cloud based solutions. In a recent post on this blog I wrote about Three Remarkable Observations about Reltio. The fourth will be that this is a cloud-based MDM (and more) solution – called Reltio Cloud.

Another example of going the cloud path is Riversand. Their new release is put forward as a cloud-native suite of Master Data Management solutions as told in an interview by Katie Fabiszak with CEO & Founder Upen Varanasi of Riversand. The interview is posted as a guest blog post on The Disruptive MDM List. The post is called Cloud multi-domain MDM as the foundation for Digital Transformation.

Cloud MDM

 

 

The Three MDM Ages

Master Data Management (MDM) is relatively new discipline. The future will prove what is was, but standing here in mid-2018 I see that we already had 2 ages and are now slowly proceeding into a 3rd age. These ages can be coined as:

  • Pre MDM,
  • Middle MDM and
  • High MDM

Pre MDM

In these dark ages the term Master Data Management may have been used, but there were not any established discipline, methodologies, frameworks and technology solutions around that truly could count as MDM.

We had Customer Data Integration (CDI) around, we had Product Information Management (PIM) in the making and some of us were talking Data Quality Management – and that in practice being namely deduplication / data matching.

Middle MDM

MDM as Three Letter Acronym (TLA) emerged in the mid 00’s as told in the post Happy 10 Years Birthday MDM Solutions.

It was at that time Aaron Zornes changed his stage name from The Customer Data Integration Institute to The MDM Institute.

During this age many MDM solutions slowly but steadily have developed into multi-domain MDM solutions as reported over at the Disruptive MDM List in the blog post called 4 Vendor Paths to Multidomain MDM covering the road travelled by 10 vendors on the MDM market.

Most MDM solutions in the Middle MDM Age have been deployed on-premise

High MDM

We are now cruising into the High MDM Age. First and foremost a lot more organizations are now implementing MDM. Many new deployments are cloud based. New ways are tried out like encompassing more than master data in the same platform.

The jury is of course still out about what will be some main trends of the High MDM Age. My money is placed on what Gartner, the analyst firm, calls Multienterprise MDM as elaborated in the post Ecosystem Wide MDM.

MDM Ages.png

Welcome Dynamicweb PIM on the Disruptive MDM and PIM List

This Disruptive Master Data Management Solutions list is a list of available:

  • Master Data Management (MDM) solutions
  • Customer Data Integration (CDI) solutions
  • Product Information Management (PIM) solutions
  • Digital Asset Management (DAM) solutions.

You can use this site as a supplement to the likes of Gartner, Forrester, MDM Institute and others when selecting a MDM / CDI / PIM / DAM solution, not at least because this site will include both larger and smaller disruptive MDM, PIM and similar solutions.

The latest entry on the list is Dynamicweb PIM. This is a mature cloud-based Product Information Management (PIM) solution that can be deployed either as a stand-alone PIM implementation or in their combined all-in-one platform together with content management, ecommerce and marketing and tightly integrated with popular ERP and CRM solutions. This integrated approach offers a short time to value opportunity for midsized companies on the quest to ramp up online sales.

Read more about the Dynamicweb PIM solution here.

Dynamicweb PIM front

Three Remarkable Observations about Reltio

The latest entry on The Disruptive Master Data Management Solutions List is Reltio. I have been following Reltio for more than 5 years and have had the chance to do some hands on lately.

In doing that, I think there are three observations that makes the Reltio Cloud solution a remarkable MDM offering.

More than Master Data

While the Reltio solution emphasizes on master data the platform can include the data that revolves around master data as well. That means you can bring transactions and big data streams to the platform and apply analytics, machine learning, artificial intelligence and those shiny new things in order to go from a purely analytical world for these disciplines to exploit these data and capabilities in the operational world.

The thinking behind this approach is that you can not get a 360-degree on customer, vendor and other party roles as well as 360-degree on products by only having a snapshot compound description of the entity in question. You also need the raw history, the relationships between entities and access to details for various use cases.

In fact, Reltio provides not just operational MDM, but through a module called Reltio IQ also brings continuously mastered data, correlated transactions into an Apache Spark environment for analytics and Machine Learning. This eliminates the traditional friction of synchronizing data models between MDM and analytical environments. It also allows for aggregated results to be synchronized back into the MDM profiles, by storing them as analytical attributes. These attributes are now available for use in operational context, such as marketing segmentation, sales recommendations, GDPR exposure and more.

Multiple Storing Capabilities

There is an ongoing debate in the MDM community these days about if you should use relational database technology or NoSQL technology or graph technology? Reltio utilizes all three of them for the purposes where each approach makes the most sense.

Reference data are handled as relational data. The entities are kept using a wide column store, which is a technique encompassing scalability known from pure column stores but with some of the structure known from relational databases. Finally, the relationships are handled using graph techniques, which has been a recurring subject on this blog.

Reltio calls this multi-model polyglot persistence, and they embrace the latest technologies from multiple clouds such as AWS and Google Cloud Platform (GCP) under the covers.

Survival of the Fit Enough

One thing that MDM solutions do is making a golden record from different systems of records where the same real-world entity is described in many ways and therefore are considered duplicate records. Identifying those records is hard enough. But then comes the task of merging the conflicting values together, so the most accurate values survive in the golden record.

Reltio does that very elegantly by actually not doing it. Survivorship rules can be set up based on all the needed parameters as recency, provenance and more and you may also allow more than one value to survive as touched in the post about the principle of Survival of the Fit Enough.

In Reltio there is no purge of the immediately not surviving values. The golden record is not stored physically. Instead Reltio keeps one (or even more than one) virtual golden record(s) by letting the original source records stay. Therefore, you can easily rollback or update the single view of the truth.

The Reltio platform allows survivorship rules to be customized in rulesets for an unlimited number of roles and personas. In effect supporting multiple personalized versions of the truth. In an operational MDM context this allows sales, marketing, compliance, and other teams to see the data values that they care about most, while collaborating continuously in what Reltio calls the Self-Learning Enterprise.

Going beyond operational MDM

 

Product Data Lake Behind the Scenes

Product Data Lake is a cloud service for exchanging product information (product data syndication) between manufacturers, distributors and merchants. When telling about the service I usually concentrate on the business benefits and how the service will make you sell more and reduce costs.

However, there will always be one or two persons in the audience who wants to know about the technology behind. And for sure, this is important too.

The service is built using some of the newest and best-of-breed technologies available for this purpose today. This includes Amazon Elastic Computing Cloud for hosting the public cloud version, MongoDB for storing data, RabbitMQ for handling data streams and ElasticSearch for finding data.

PDL Architecture

You can dive into the geeky parts in this PDF document: Product Data Lake Architecture.

MDM in The Cloud, On-Premise or Both

One of the forms of Master Data Management (MDM) is the rising cloud deployment model as touched in the Disruptive MDM List blog post about 8 Forms of Master Data Management.

If we look at the MDM solution vendors, they may in that sense be divided into three kinds:

  • Cloud only, which are vendors born in the cloud age and who are delivering their service in the cloud only. Reltio is an example of that kind of MDM vendor.
  • Cloud or on-premise, which are vendors that can deliver both in the cloud and on premise, but where it makes most sense that you as a customer chooses the one that fits you the best. An example is Semarchy.
  • Cloud and on-premise. Informatica is the example of an MDM vendor that embraces both deployment models (together with other data management disciplines) at the same time (called hybrid) as told in an article by Kristin Nicole of SiliconANGLE. The title goes like this: Balancing act: Informatica straddles on-prem needs with cloud data at Informatica World 2018

Cloud MDM

A Business Oriented Data Mind Map

You can look at data in many ways.

Below is a mind map embracing some of the ways you can make a picture of data within your business.

data mind map

Data is often seen as the raw material that will be processed into information, which can be used to gather knowledge and thereby over time emerge as business wisdom.

When working with processing data we may distinguish between structured data that is already pre-processed into a workable format and unstructured data that is not easily ingested as information yet.

The main forms of structured data are:

  • Reference data that often is defined and maintained in a wider scope than in your organization but where you still may consider be more knowledgeable inside your organisation as touched in the post The World of Reference Data.
  • Master data that describes the who, where and what in your business transactions. You can drill further down into this in the post A Master Data Mind Map.
  • Transactions that holds the details of the ongoing production events, about when we make purchases and sales and the financials related to all activities in the business.

Unstructured data will in the end hold much more information than our structured data. This includes communication data, digital assets and big data. Some structured data sources are though also big as examined in the post Five Flavors of Big Data.

We may also store the data in different places. For historical reasons within computer technology we have stored our data on premise, but organizations are, in different pace, increasingly depolying new data stores in the cloud.

In organisations with activities in multiple geographies and/or other organizational splits an ongoing consideration is whether a chunk of data is to be handled locally for each unit or to be handled globally (within the organization).

I am sure there are a lot of other ways in which you can look at data. What is on your mind?