From sitting at home these are my thoughts about how data management can be changed due to the current outbreak of the Covid-19 (Corona) virus and the longer-term behaviour impact after the pandemic hopefully will be over.
Ecommerce Will Grow Faster
Both households and organizations are buying more online and this trend is increasing due to the urge of keeping a distance between humans. The data management discipline that underpins well executed ecommerce is Product Information Management (PIM). We will see more organizations implementing PIM solutions and we must see more effective and less time-consuming ways of implementing PIM solutions.
Data Governance Should Mature Faster
The data governance discipline has until now been quite immature and data governance activities have been characterized by an endless row of offline meetings. As data governance is an imperative in PIM and any other data management quest, we must shape data governance frameworks that are more ready to use, and we must have online learning resources available for both professionals and participating knowledge workers with various roles.
Data Sharing Could Develop Faster
People, organizations and countries initially act in a selfish manner during a crisis, but we must realize that collaboration including data sharing is the only way forward. Hopefully we will see more widespread data sharing enterprise wide as this will ease remote working. Also, we could see increasing interenterprise (business ecosystem wide) data sharing which in particular will ease PIM implementations through automated Product Data Syndication (PDS).
KDR Recruitment is a data management recruitment company and one of those rare recruitment agencies that genuinely express an interest in the disciplines covered.
This is manifested in among other things a yearly survey and report about the state of data that also was touched on this blog five years ago in the post Integration Matters.
This year the surveyed topics include for example how to use data analysis, new skills needed and the most effective ways to improve data quality. You can participate with your experience and observations here at State of Data 2020.
The Master Data Management (MDM) discipline is emerging. A certain trend is that MDM solutions will grow beyond handling traditional master data entities and encompass other kinds of data and more capabilities that can be used for other kinds of data as well.
- Utilizing data discovery to explore data sources with master data, reference data, critical application data and other kinds of data as described in the post How Data Discovery Makes a Data Hub More Valuable.
- Governing the full set of data that needs to be governed as examined in the post Maturing RDM, MDM and ADM With Collaborative Data Governance.
- Building a data hub that encompass the right representation of data that needs to be shared enterprise wide and even business ecosystem wide as explained in the post Why Flexible Data Models are Crucial in Data Sharing.
- Measuring data quality in conjunction with general key performance indicators in dashboards that besides master data also embraces other internal and external sources as for example aggregated data from data warehouses and data lakes.
These themes were also covered in a webinar I presented with Semarchy last month. Watch the webinar The Intelligent Data Hub: MDM and Beyond.
Data Governance and Master Data Management (MDM) are overlapping disciplines. When embarking on a data governance initiative you may encounter some difficulties in what belongs to the data governance side and what belongs to the master data side. One of the challenges is that data governance should also encompass other data than master data. The most common examples are reference data and other critical application data than master data.
So, while you may get coverage for setting up data stewardship, processes and the data platform for master data in a traditional MDM tool, other important aspects as the data governance related to Reference Data Management (RDM) and Application Data Management (ADM) may have to be implemented separately.
This calls for taking the MDM solution to the next level by encompassing reference data and application data as well. In that way essential data governance definition components as a business glossary, data policies and data standards as well as the enforcement components through data stewardship can be implemented in a collaborative way:
In this case the MDM platform will be extended to be an intelligent data hub. In collaboration with FX Nicolas I will be presenting such a solution in a webinar hosted by Semarchy. The webinar goes live Wednesday 13th November at 5pm CET / 11am ET. Register here on Intelligent Data Hub: MDM and Beyond.
Data discovery is emerging as an essential discipline in the data management space as explained in the post The Role of Data Discovery in Data Management.
In a data hub encompassing master data, reference data, critical application data and more, data discovery can play a significant role in the continuous improvement of data quality and how data is governed, managed and measured along with an ever evolving business model and new data driven services.
Data discovery serves as the weapon used when exploring the as-is data landscape at your organization with the aim of building a data hub that reflects your data model and data portfolio. As the data maturity is continuously improved reflected in step-by-step maturing to-be states, data discovery can be used when increasing the data hub scope by encompassing more data sources, when new data driven services are introduced and the business model is enhanced as part of a digital transformation.
In that way data discovery is an indispensable node in maturing the data supply chain and the continuously data quality improvement cycle that must underpin your digital transformation course.
Learn more about the data discovery capability in a data hub context in the Semarchy whitepaper authored by me and titled Intelligent Data Hub: -Taking MDM to the Next Level.
During the end of last century data quality management started to gain traction as organizations realized that the many different applications and related data stores in operation needed some form of hygiene. Data cleansing and data matching (aka deduplication) tools were introduced.
In the 00’s Master Data Management (MDM) arised as a discipline encompassing the required processes and the technology platforms you need to have to ensure a sustainable level of data quality in the master data used across many applications and data stores. The first MDM implementations were focused on a single master data domain – typically customer or product. Then multidomain MDM (embracing customer and other party master data, location, product and assets) has become mainstream and we see multienterprise MDM in the horizon, where master data will be shared in business ecosystems.
MDM also have some side disciplines as Product Information Management (PIM), Digital Asset Management (DAM) and Reference Data Management (RDM). Sharing of product information and related digital assets in business ecosystems is here supported by Product Data Syndication.
Lately data governance has become a household term. We see multiple varying data governance frameworks addressing data stewardship, data policies, standards and business glossaries. In my eyes data governance and data governance frameworks is very much about adding the people side to the processes and technology we have matured in MDM and Data Quality Management (DQM). And we need to combine those themes, because It is not all about People or Processes or Technology. It is about unifying all this.
In my daily work I help both tool providers and end user organisations with all this as shown on the page Popular Offerings.
A Request for Proposal (RFP) process for a Master Data Management (MDM) and/or Product Information Management (PIM) solution has a hard fact side as well as there are The Soft Sides of MDM and PIM RFPs.
The hard fact side is the detailed requirements a potential vendor has to answer to in what in most cases is the excel sheet the buying organization has prepared – often with the extensive help from a consultancy.
Here are what I have seen as the most frequently included topics for the hard facts in such RFPs:
- MDM and PIM: Does the solution have functionality for hierarchy management?
- MDM and PIM: Does the solution have workflow management included?
- MDM and PIM: Does the solution support versioning of master data / product information?
- MDM and PIM: Does the solution allow to tailor the data model in a flexible way?
- MDM and PIM: Does the solution handle master data / product information in multiple languages / character sets / script systems?
- MDM and PIM: Does the solution have capabilities for (high speed) batch import / export and real-time integration (APIs)?
- MDM and PIM: Does the solution have capabilities within data governance / data stewardship?
- MDM and PIM: Does the solution integrate with “a specific application”? – most commonly SAP, MS CRM/ERPs, SalesForce?
- MDM: Does the solution handle multiple domains, for example customer, vendor/supplier, employee, product and asset?
- MDM: Does the solution provide data matching / deduplication functionality and formation of golden records?
- MDM: Does the solution have integration with third-party data providers for example business directories (Dun & Bradstreet / National registries) and address verification services?
- MDM: Does the solution underpin compliance rules as for example data privacy and data protection regulations as in GDPR / other regimes?
- PIM: Does the solution support product classification and attribution standards as eClass, ETIM (or other industry specific / national standards)?
- PIM: Does the solution support publishing to popular marketplaces (form of outgoing Product Data Syndication)?
- PIM: Does the solution have a functionality to ease collection of product information from suppliers (incoming Product Data Syndication)?
Learn more about how I can help in the blog page about MDM / PIM Tool Selection Consultancy.
Data discovery is a term probably most mentioned in relation to business intelligence and data science. I this context data discovery can be seen as a more experimental and preliminary activity that can lead to a more continuous and integrated form of reporting and predictive analysis when hidden data sources, relationships and patterns are identified.
However, data discovery is useful in other data management disciplines as well.
With the increasing awareness of data security, data protection and data privacy – and the regularity compliance enforced in this space – it is crucial for organisations to know what kind of data that flows and are stored within the organization. While you may argue that this should be available in already existing documentation, I have yet to meet an organization, where this is the case. And I come around a lot.
Data discovery is also a component of test data management and tool vendors package their offerings in this space with capabilities for data masking, data subsetting and data discovery in order to answer questions as:
- Where are the data elements that should be masked when using production data in test scenarios without violating data privacy regulations?
- How can you subset (minimize) test data sets derived from production (covering several databases) and still have proper relationships covered?
Within Data Quality Management, Data Governance and Master Data Management (MDM) data discovery also plays a role similar to the role in data reporting. We can use data discovery to map data lineage, find potential data relationships where data matching, data cleansing and/or data stewardship might help with ensuring data quality and business process improvement and explore where the same data have different labels (metadata) attached or the same labels are used for different data types.
Yesterday I had the pleasure of attending the Informatica MDM 360 and Data Governance Summit in London including being in a panel discussing best practices for your MDM 360 journey. The rise of Artificial Intelligence (AI) in Master Data Management (MDM) was a main theme at this event.
Informatica has a track record of innovating in new technologies in the data management space while also acquiring promising newcomers in order to fast track their market offering. So it is with AI and MDM at Informatica too. Informatica currently has two tracks:
- clAIre – the clairvoyant component in the Informatica portfolio that “using machine learning and other AI techniques leverages the industry-leading metadata capabilities of the Informatica Intelligent Data Platform to accelerate and automate core data management and governance processes”.
- Informatica Customer 360 Insights which is the new branding of the recent AllSight acquisition. You can learn about that over at The Disruptive Master Data Management Solutions List in the entry about Informatica Customer 360 Insights.
At the Informatica event the synergy between these two tracks was presented as the Intelligent 360 View. Naturally, marketing synergies are the first results of an acquisition. Later we will – hopefully – see actual synergies when the technologies are to be aligned, positioned and delivered to customers who want to be an intelligent enterprise of the future.
The title of this blog post is also the title of a presentation I will do at the 2019 Data Governance and Information Quality Conference in San Diego, US in June.
There is a little difference between how we can exercise data governance and information quality management when we are handling data about products versus handling the most common data domain being party data (customer, vendor/supplier, employee and other roles).
This topic was touched here on the blog in the post called Data Quality for the Product Domain vs the Party Domain.
The conference session will go through these topics:
- Product master data vs. product information
- How Master Data Management (MDM), Product Information Management (PIM) and Digital Asset Management (DAM) stick together
- The roles of 1st party data, 2nd party data and 3rd party data in MDM, PIM and DAM
- Business ecosystem wide product data management
- Cross company data governance and information quality alignment
You can have a look at the full agenda for the DGIQ 2019 Conference here.