The Emperor´s New Term

Emperor_Clothes“No one dared to admit that he couldn’t see anything, for who would want it to be known that he was either stupid or unfit for his post?”

This is a quote from the story called The Emperors New Clothes by Hans Christian Andersen.

Having been in and around the IT business for nearly 40 years I have seen, and admittedly not seen, a lot. Inflated hype has always been there, and a lot of technologies, companies and gurus did not make it, but came out naked.

What will you say are the emperor’s new clothes within data management today. Here are some suggestions:

  • Social MDM (Social Master Data Management): The idea that master data management will embrace social profiles and social data streams. If not anything else, did GDPR kill that one?
  • Big Data: This term has been killed so many times. But were those always a staged murder?
  • Single source of truth: The vision that we can have one single source that encompasses everything we need to know about a business entity. This has been a long time running question. Will it ever be answered?

What is your suggestion?

A Business Oriented Data Mind Map

You can look at data in many ways.

Below is a mind map embracing some of the ways you can make a picture of data within your business.

data mind map

Data is often seen as the raw material that will be processed into information, which can be used to gather knowledge and thereby over time emerge as business wisdom.

When working with processing data we may distinguish between structured data that is already pre-processed into a workable format and unstructured data that is not easily ingested as information yet.

The main forms of structured data are:

  • Reference data that often is defined and maintained in a wider scope than in your organization but where you still may consider be more knowledgeable inside your organisation as touched in the post The World of Reference Data.
  • Master data that describes the who, where and what in your business transactions. You can drill further down into this in the post A Master Data Mind Map.
  • Transactions that holds the details of the ongoing production events, about when we make purchases and sales and the financials related to all activities in the business.

Unstructured data will in the end hold much more information than our structured data. This includes communication data, digital assets and big data. Some structured data sources are though also big as examined in the post Five Flavors of Big Data.

We may also store the data in different places. For historical reasons within computer technology we have stored our data on premise, but organizations are, in different pace, increasingly depolying new data stores in the cloud.

In organisations with activities in multiple geographies and/or other organizational splits an ongoing consideration is whether a chunk of data is to be handled locally for each unit or to be handled globally (within the organization).

I am sure there are a lot of other ways in which you can look at data. What is on your mind?

 

Seven Flavors of MDM

Master Data Management (MDM) can take many forms. An exciting side of being involved in MDM implementations is that every implementation is a little bit different which also makes room for a lot of different technology options. There is no best MDM solution out there. There are a lot of options where some will be the best fit for a given MDM implementation.

The available solutions also change over the years – typically by spreading to cover more land in the MDM space.

In the following I will shortly introduce the basic stuff with seven flavours of MDM. A given MDM implementation will typically be focused on one of these flavours with some elements of the other flavors and a given piece of technology will have an origin in one of these flavours and in more or less degree encompass some more flavors.

7 flavours

The traditional MDM platform

A traditional MDM solution is a hub for master data aiming at delivering a single source of truth (or trust) for master data within a given organization either enterprise wide or within a portion of an enterprise. The first MDM solutions were aimed at Customer Data Integration (CDI), because having multiple and inconsistent data stores for customer data with varying data quality is a well-known pain point almost everywhere. Besides that, similar pain points exist around vendor data and other party roles, product data, assets, locations and other master data domains and dedicated solutions for that are available.

Product Information Management (PIM)

Special breed of solutions for Product Information Management aimed at having consistent product specifications across the enterprise to be published in multiple sales channels have been around for years and we have seen a continuously integration of the market for such solutions into the traditional MDM space as many of these solutions have morphed into being a kind of MDM solution.

Digital Asset Management (DAM)

Not at least in relation to PIM we have a distinct discipline around handling digital assets as text documents, audio files, video and other rich media data that are different from the structured and granular data we can manage in data models in common database technologies. A post on this blog examines How MDM, PIM and DAM Stick Together.

Big Data Integration

The rise of big data is having a considerable influence on how MDM solutions will look like in the future. You may handle big data directly inside MDM og link to big data outside MDM as told in the post about The Intersection of MDM and Big Data.

Application Data Management (ADM)

Another area where you have to decide where master data stops and handling other data starts is when it comes to transactional data and other forms data handled in dedicated applications as ERP, CRM, PLM (Product Lifecycle Management) and plenty of other industry specific applications. This conundrum was touched in a recent post called MDM vs ADM.

Multi-Domain MDM

Many MDM implementations focus on a single master data domain as customer, vendor or product or you see MDM programs that have a multi-domain vision, overall project management but quite separate tracks for each domain. We have though seen many technology vendors preparing for the multi-domain future either by:

  • Being born in the multi-domain age as for example Semarchy
  • Acquiring the stuff as for example Informatica and IBM
  • Extend from PIM as for example Riversand and Stibo Systems

MDM in the cloud

MDM follows the source applications up into the cloud. New MDM solutions naturally come as a cloud solution. The traditional vendors introduce cloud alternatives to or based on their proven on-promise solutions. There is only one direction here: More and more cloud MDM – also as customer as business partner engagement will take place in the cloud.

Ecosystem wide MDM

Doing MDM enterprise wide is hard enough. But it does not stop there. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus we will have a need for working on the same foundation around master data as reported in the post Ecosystem Wide MDM.

Why it is not a Product Data Warehouse, but a Product Data Lake

There is a need for a new solution to sharing product information between trading partners. Product Data Lake is that new solution. Using the term data lake as a part of the name for the solution is very deliberate. Here is why:

Volume

When setting up a warehouse, and a data warehouse, you have to estimate the storing size and the throughput. There will be a limit to how much data you can store and how much data you can upload and download within a given period.

Our vision is that Product Data Lake will be the process driven key service for exchanging any sort of product information within business ecosystems all over the world, with the aim of optimally assist self-service purchase of every kind of product.

In order to achieve that vision, we need to be able to scale up drastically. Therefore, we use a document-oriented database called MongoDB to store product information.

Even if you choose to implement a Product Data Lake instance for a single business ecosystem, you will benefit from the high scalability.

Velocity

Business ecosystems changes all the time. You need to rapidly be able to adapt your data management, not at least when it comes to exchanging product information.

Swapping trading partners is one thing. That often means dealing with other product information requirements and opportunities and adhering to other standards.

We will also see business ecosystems in new shapes in the future. There will be fewer nodes between manufacturers and point-of-sales and point-of-sales will more likely be online marketplaces.

However, the changes will not happen as a big bang but in varying pace for each industry, geography and organization.

The rigid consensus structure of a data warehouse, and product information exchange solutions that resembles a data warehouse, will not cope with that change. The data lake concept, in the form of Product Data Lake, will.

In Product Data Lake you as a provider upload product information in your structure and format and you as a receiver download in your structure and format. The linking and transformation takes place inside Product Data Lake using linked metadata.

Variety

While everyone agrees that a common standard for all product information is the best answer we must on the other hand accept, that using a common standard for every kind of product and every piece of information needed is quite utopic. We haven’t even a common uniquely spelled term in English for standardization/standarisation.

Also, we must foresee that one organization will mature in a different pace than another organisation in the same business ecosystem.

These observations are the reasons behind the launch of Product Data Lake. In Product Data Lake we encompass the use of (in prioritized order):

  • The same standard in the same version
  • The same standard in different versions
  • Different standards
  • No standards
Learn about some of these standards in the post Five Product Classification Standards.
big data pdl.png

Data Lakes in Business Ecosystems

The concept of a data lake has until now mainly been used to describe how data gathered by a given organization can be organized in order to provide for analytical purposes. The data lake concept is closely tied to the term big data, which means that a data lake caters for handling huge volumes of data, with high velocity and where data comes in heaps of varieties. A data lake is much more agile than data marts and data warehouses, where you have to determine the purpose of the use of data beforehand.

In my eyes the idea about that every organization should gather all the data of interest behind its own firewall does not make sense. Some data will for sure be only for eyes of people behind the corporate walls. Here you should indeed put your private data into your own data lake. But a lot of data are common known data that everyone should not spend time on collecting and eventually cleansing. Here you should collaborate within your industry or other sphere around data lakes for public data.

Perhaps most importantly you should share data lakes with other members of your business ecosystem. You probably already do share data within business ecosystems with your trading partners. Until now such sharing has resembled the concepts of data marts and data warehouses being very purpose specific exchanges of data build on common beforehand understood standards for data.

Right now I am working on a cloud service called the Product Data Lake. Here we host a data lake for sharing product data in the business ecosystems of manufacturers, distributors, merchants and large end users of product information. You can join our journey by following us here on LinkedIn at the Product Data Lake LinkedIn Company Page.

bridge into lake.png

The 360 Ways to Improving Customer Experiences

In today’s blog post over on The Disruptive Master Data Management Solutions List the CEO of AllSight, David Corrigan, examines 3 Reasons MDM No Longer Delivers a Customer 360.

In here David explores the topics in the new era of the customer 360 degree view being encompassing all customer data, covering analytical and operational usages and improving customer experience.

The post includes this testimonial from Deotis Harris, Senior Director, MDM at Dell EMC: “We saw an opportunity to leverage AllSight’s modern technology (Customer Intelligence), coupled with our legacy systems such as Master Data Management (MDM), to provide the insight required to enable our sellers, marketers and customer service reps to create better experiences for our customers.”

By the way: Being a MDM practitioner who have spent many years with customer 360 and now spending equal chunks of time with product 360, I find the forward-looking topics being very similar between customer 360 and product 360. In short:

  • The span of product data to handle has increased dramatically in recent years as told in the post Self-service Ready Product Data.
  • We can use the same data architecture for analytical and operational purposes as mentioned in the post The Intersection of MDM and Big Data.
  • It is all about creating better experiences for your customers.

360

 

The Intersection of MDM and Big Data

Back in 2015 Gartner, within a Magic Quadrant for MDM, described two different ways observed in how you may connect big data and master data management as reported in the post Two Ways of Exploiting Big Data with MDM.

In short, the two ways observed were:

  • Capabilities to perform MDM functions directly against copies of big data sources such as social network data copied into a Hadoop environment. Gartner then found that there have been very few successful attempts (from a business value perspective) to implement this use case, mostly as a result of an inability to perform governance on the big datasets in question.
  • Capabilities to link traditionally structured master data against those sources. Gartner then found that this use case is also sparse, but more common and more readily able to prove value. This use case is also gaining some traction with other types of unstructured data, such as content, audio and video.

In my eyes the ability to perform governance on big datasets is key. In fact, master data will tend to be more externally generated and maintained, just like big data usually is. This will change our ways of doing information governance as for example discussed in the post MDM and SCM: Inside and outside the corporate walls.

Eventually, we will see use cases of intersections of MDM and big data. The one I am working with right now is about how you can improve sharing of product master data (product information) between trading partners. While this quest may be used for analytical purposes, which is the said aim with big data, this service will fundamentally serve operational purposes, which is the predominant aim with master data management.

This big data, or rather data lake, approach is about how we by linking metadata connects different perceptions of product information that exists in cross company supply chains. While everyone being on the same standard at the same time would be optimal, this is quite utopic. Therefore, we must encourage pushing product information (including rich textual content, audio and video) with the provider’s standard and do the “schema-on-read” stuff when each of the receivers pulls the product information for their purposes.

If you want to learn more about how that goes, you can follow Product Data Lake here.

MDM and Big Data

Customer Insight vs Product Insight

The rise of big data is very much driven by a craving for getting more insight on your (prospective) customers. However, the coin has a (better) flip side.

Looking at it from the other side

As a customer, we will strike back. We do not need to be told what to buy. But we do want to know what we are buying. This means we want to be able to see rich product information when making a self-service purchase. This subject was examined in the post You Must Supplement Customer Insight with Rich Product Data.

Many companies who are involved in selling to private and business customers are ramping up maintenance of product data by implementing inhouse Product Information Management (PIM) solutions as told in yesterday’s guest post on this blog. The article is called The Relation of PIM to Retail Success.

One further challenge is that you have to get product information from the source, usually being the manufacturers.

Big data approaches work for both

As data lakes are used to being the place to harvest customer insight, the data lake concept can be the approach to provide product insight to end customers as well.

The problem with having product data flowing from manufacturers to distributors and merchants is that everyone does not use the same standard, format, structure and taxonomy for product information.

The solution is a data lake shared by the business ecosystem. It is called Product Data Lake.

Sell more Reduce costs

 

Passive vs Active Product Information Exchange

Product information is the kind of data that usually flows cross company. The most common routes start with that the hard facts about a product originates at the manufacturer. Then the information may be used on the brands own website, propagated to a marketplace (online shop-in-shop) and also propagated downstream to distributors and merchants.

The challenge to the manufacturer is that this represent many different ways of providing product information, not at least when it comes to distributors and merchants, as these will require different structures and formats using various standards and not being on the same maturity level.

Looking at this from the downstream side, the distributors and merchants, we have the opposite challenge. Manufacturers provide product information in different structurers and formats using various standards and are not on the same maturity level.

Supply chain participants can challenge this in a passive or an active way. Unfortunately, many have chosen – or are about to choose – the passive way. It goes like this:

  • As a manufacturer, we have a product data portal where trading partners who wants to do business with us, who obviously is the best manufacturer in our field, can download the product information we have in our structure and format using the standards we have found best.
  • As a distributor/merchant we have a supplier product data portal where trading partners who wants to do business with us, the leading player in our field, can upload the product information we for the time being will require in our structure and format using the standard(s) we have found best.

Passive vs ActiveThis approach seems to work if you are bigger than your trading partner. And many times one will be bigger than the other. But unless you are very big, you will in many cases not be the biggest. And in all cases where you are the biggest, you will not be seen as a company being easy to do business with, which eventually will decide how big you will stay.

The better way is the active way creating a win-win situation for all trading partners as described in the article about Product Data Lake Business Benefits.