During the end of last century data quality management started to gain traction as organizations realized that the many different applications and related data stores in operation needed some form of hygiene. Data cleansing and data matching (aka deduplication) tools were introduced.
In the 00’s Master Data Management (MDM) arised as a discipline encompassing the required processes and the technology platforms you need to have to ensure a sustainable level of data quality in the master data used across many applications and data stores. The first MDM implementations were focused on a single master data domain – typically customer or product. Then multidomain MDM (embracing customer and other party master data, location, product and assets) has become mainstream and we see multienterprise MDM in the horizon, where master data will be shared in business ecosystems.
MDM also have some side disciplines as Product Information Management (PIM), Digital Asset Management (DAM) and Reference Data Management (RDM). Sharing of product information and related digital assets in business ecosystems is here supported by Product Data Syndication.
Lately data governance has become a household term. We see multiple varying data governance frameworks addressing data stewardship, data policies, standards and business glossaries. In my eyes data governance and data governance frameworks is very much about adding the people side to the processes and technology we have matured in MDM and Data Quality Management (DQM). And we need to combine those themes, because It is not all about People or Processes or Technology. It is about unifying all this.
In my daily work I help both tool providers and end user organisations with all this as shown on the page Popular Offerings.
A Request for Proposal (RFP) process for a Master Data Management (MDM) and/or Product Information Management (PIM) solution has a hard fact side as well as there are The Soft Sides of MDM and PIM RFPs.
The hard fact side is the detailed requirements a potential vendor has to answer to in what in most cases is the excel sheet the buying organization has prepared – often with the extensive help from a consultancy.
Here are what I have seen as the most frequently included topics for the hard facts in such RFPs:
MDM and PIM: Does the solution have functionality for hierarchy management?
MDM and PIM: Does the solution have workflow management included?
MDM and PIM: Does the solution support versioning of master data / product information?
MDM and PIM: Does the solution allow to tailor the data model in a flexible way?
MDM and PIM: Does the solution handle master data / product information in multiple languages / character sets / script systems?
MDM and PIM: Does the solution have capabilities for (high speed) batch import / export and real-time integration (APIs)?
MDM and PIM: Does the solution have capabilities within data governance / data stewardship?
MDM and PIM: Does the solution integrate with “a specific application”? – most commonly SAP, MS CRM/ERPs, SalesForce?
MDM: Does the solution handle multiple domains, for example customer, vendor/supplier, employee, product and asset?
MDM: Does the solution provide data matching / deduplication functionality and formation of golden records?
MDM: Does the solution have integration with third-party data providers for example business directories (Dun & Bradstreet / National registries) and address verification services?
MDM: Does the solution underpin compliance rules as for example data privacy and data protection regulations as in GDPR / other regimes?
PIM: Does the solution support product classification and attribution standards as eClass, ETIM (or other industry specific / national standards)?
PIM: Does the solution support publishing to popular marketplaces (form of outgoing Product Data Syndication)?
PIM: Does the solution have a functionality to ease collection of product information from suppliers (incoming Product Data Syndication)?
The latest Gartner MDM Magic Quadrant mentions the 2017 revenues as estimated by Gartner:
It is worth noticing that Oracle is not a Gartner MDM Magic Quadrant vendor anymore and the Gartner report indicate that Oracle still have an MDM (or is it ADM?) revenue from the installed base resembling the ones of the other mega-vendors being SAP, IBM and Informatica.
Update: The revenues mentioned are assumed to be software license and maintenance. The vendors may then have additional professional services revenue.
The 14 MDM vendors that qualified for inclusion in the latest quadrant constituted, according to Gartner estimates, 84% of the estimated MDM market revenue (software and maintenance) for 2017 – which according to Gartner criteria must be excluding Oracle.
Providing a 360-degree view of master data entities
Enabling happy self-service scenarios
Underpinning the best customer experience
Encompassing Internet of Things (IoT)
Providing a 360-degree view of master data entities through Golden Records in Multidomain MDM will be much easier by sharing master data that is already digitalised as third-party reference data and/or at business partners.
Enabling happy self-service scenarios can be done much more effectively by opening up the master data onboarding to business partners and customers them selves and by letting product data flow easily between trading partners as pondered in the post Linked Product Data Quality.
Underpinning the best customer experience will require that you utilize data from and about the whole business ecosystem where your company is a participant.
Encompassing Internet of Things (IoT) means that you must share master within the business ecosystem as touched in the post IoT and MDM.
In the latest Gartner Master Data Management Solutions magic quadrant it is stated that “Gartner-estimated 2016 revenues of the four largest vendors commanded over 77% of the market (SAP, IBM, Oracle and Informatica). These four held a Gartner-estimated market share of just over 73% in 2017.”
There seems to be a little decrease in the dominance from the megavendors, though the market according to Gartner is still ruled by the big four. The number of licenses sold by these vendors and those midsize vendors who also are in the quadrant can be found in the post Counting MDM Licenses.
For the future trend it is also worth noticing, that Oracle is not a part of the MDM magic quadrant anymore, as Oracle in the Gartner lingo is not an MDM vendor, but an ADM vendor today. Oracle is not included in the latest Forrester MDM wave either.
Another market distinction is around MDM versus Product Information Management (PIM) solutions. The post Several Sources of Truth about MDM / PIM Solutions examines the positioning by Gartner and Forrester and in that sense the magavendors are better at MDM than PIM.
Any Gartner estimation will be biased towards large vendors having large clients as these are the Gartner clients. In a LinkedIn discussion a big four person suggested that the market is fragmented and there are many MDM-like solutions.
This remark followed the market estimation from a fresh market report from Information Difference. The positioning results from here were shown in the post Movements in the MDM Vendor Landscape 2019. In here the megavendors did not perform so well on the technology axis, which is largely made up by customer satisfaction feedback in the underlying survey.
So, what do you think? Will the megavendors still rule the MDM market or will the midsize and smaller vendors get a larger piece of the cake?
However, before reading too much I was prompted with an inescapable form asking for my details in a master data sharing tone.
Well, then I could as well explore the mandatory country list. No surprise. A master (or reference) data havoc. Two Bosnia (and) Herzegovina entries. Two Brunei entries. Two Brazil entries. Two Burma / Myanmar entries.
Apart from positioning some of the tool vendors on a chart, the report also estimates the size of the MDM market. Information difference estimates that the software vendors make 1.6 B USD per year. Hereof are pure license sales 885 M USD, maintenance fees are 273 M USD per year and professional services counts for 450 M USD per year.
In addition, the report says: “Our research shows that on average the people costs of a MDM project are four times that of the software license cost, so there is clearly a large and separate consultancy market associated with MDM”.
So, the additional spending might be in the area of 3.5 B USD (depending on how you calculate and if that multiplier is right). These costs go to system integrators, freelance MDM consultants and internal staff. From my experience internal staff are sparsely represented in MDM implementations, so yes, there is a large consultancy market within MDM.
The total 5 Billion USD spend by end user organizations yearly on MDM then look like this:
The good question that follows will of course be on the size and distribution of the business benefits achieved.
Some vendors from last year as Contentserv and Semarchy is missing this year. This is in my thinking not because they have left the marked or have become irrelevant. I will guess it is about unwillingness to contribute to the research at too many market researchers.
Profisee is added which is in line with an increased Master Data Management market exposure from the folks at Profisee.
At the recurring vendors there is little movement except that Viamedici has moved a bit up on the technology axis and Informatica (perhaps surprisingly) has fallen a bit on that axis.
Orchestra Networks is now named Ticbo following the take over since last landscape.
Data discovery is a term probably most mentioned in relation to business intelligence and data science. I this context data discovery can be seen as a more experimental and preliminary activity that can lead to a more continuous and integrated form of reporting and predictive analysis when hidden data sources, relationships and patterns are identified.
With the increasing awareness of data security, data protection and data privacy – and the regularity compliance enforced in this space – it is crucial for organisations to know what kind of data that flows and are stored within the organization. While you may argue that this should be available in already existing documentation, I have yet to meet an organization, where this is the case. And I come around a lot.
Data discovery is also a component of test data management and tool vendors package their offerings in this space with capabilities for data masking, data subsetting and data discovery in order to answer questions as:
Where are the data elements that should be masked when using production data in test scenarios without violating data privacy regulations?
How can you subset (minimize) test data sets derived from production (covering several databases) and still have proper relationships covered?
Within Data Quality Management, Data Governance and Master Data Management (MDM) data discovery also plays a role similar to the role in data reporting. We can use data discovery to map data lineage, find potential data relationships where data matching, data cleansing and/or data stewardship might help with ensuring data quality and business process improvement and explore where the same data have different labels (metadata) attached or the same labels are used for different data types.
Master Data Management (MDM) and Product Information Management (PIM) solutions are, as many other enabling technologies, often purchased using a formal Request for Proposal (RFP) process.
In such a process the theory is that the buying organization, often with the help from an external consultancy, states the functional and non-functional requirements, measure the solutions against these requirements using a weighted score model and then objectively selects the solution with the highest score.
In practice there are in my experience some more subjective and soft sides to this process. This is not at least the case in the longlist/shortlist phase and when taking the final decision. I have seen my share of overruling the scoring.
One aspect is the geographical presence of the vendor. This includes where the solution provider is based and of course also the presence around the world through local offices and partner network as told in the post Mapping MDM and PIM Solutions.
Another aspect is the subject matter expertise shown by the vendor. This includes written material provided but also available on website and blogs and of course during presentations. An example could be the emphasis on master data versus product information as exemplified in the post MDM, PIM or Both.
I have had the fortunate opportunity of being at both sides of the table during the years and are still doing that as shown in the article about Popular Offerings.