Month: April 2018

A Business Oriented Data Mind Map

Henrik Gabs Liliendahl2 Comments

You can look at data in many ways.

Below is a mind map embracing some of the ways you can make a picture of data within your business.

data mind map

Data is often seen as the raw material that will be processed into information, which can be used to gather knowledge and thereby over time emerge as business wisdom.

When working with processing data we may distinguish between structured data that is already pre-processed into a workable format and unstructured data that is not easily ingested as information yet.

The main forms of structured data are:

  • Reference data that often is defined and maintained in a wider scope than in your organization but where you still may consider be more knowledgeable inside your organisation as touched in the post The World of Reference Data.
  • Master data that describes the who, where and what in your business transactions. You can drill further down into this in the post A Master Data Mind Map.
  • Transactions that holds the details of the ongoing production events, about when we make purchases and sales and the financials related to all activities in the business.

Unstructured data will in the end hold much more information than our structured data. This includes communication data, digital assets and big data. Some structured data sources are though also big as examined in the post Five Flavors of Big Data.

We may also store the data in different places. For historical reasons within computer technology we have stored our data on premise, but organizations are, in different pace, increasingly depolying new data stores in the cloud.

In organisations with activities in multiple geographies and/or other organizational splits an ongoing consideration is whether a chunk of data is to be handled locally for each unit or to be handled globally (within the organization).

I am sure there are a lot of other ways in which you can look at data. What is on your mind?

 

The Cases for Data Matching in Multi-Domain MDM

Henrik Gabs LiliendahlLeave a comment

Data matching has always been a substantial part of the capabilities in data quality technology and have become a common capability in Master Data Management (MDM) solutions.

We use the term data matching when talking about linking entities where we cannot just use exact keys in databases.

The most prominent example around is matching names and addresses related to parties, where these attributes can be spelled differently and formatted using different standards but do refer to the same real-world entity. Most common scenarios are deduplication, where we clean up databases for duplicate customer, vendor and other party role records and reference matching, where we identify and enrich party data records with external directories.

A way to pre-process party data matching is matching the locations (addresses) with external references, which has become more and more available around the world, so you have a standardized address in order to reduce the fuzziness. In some geographies you can even make use of more extended location data, as whether the location is a single-family house, a high-rise building, a nursing home or campus. Geocodes can also be brought into the process.

matching MDMHandling the location as a separate unique entity can also be used in many industries as utility, telco, finance, transit and more.

For product data achieving uniqueness usually is a lesser pain point as told in the post Multi-Domain MDM and Data Quality Dimensions. But for sure requirements for matching products arises from time to time.

In the old days this was quite difficult as you often only had a product description that had to be parsed into discrete elements as examined in the post Matching Light Bulbs.

With the rise of Product Information Management (PIM) we now often do have the product attributes in a granular form. However, using traditional matching technology made for party master data will not do the trick as this is a different and more complex scenario. My thinking is that graph technology will help as touched in the post Three Ways of Finding a Product.

Trending Topic: Graph and MDM

Henrik Gabs Liliendahl2 Comments

Using graph data stores and utilizing the related capabilities has become a trending topic in the Master Data Management (MDM) space. This opportunity was first examined 5 years ago here on the blog in the post Will Graph Databases become Common in MDM? It seems so.

Recently David Borean, Chief Data Science Officer at the disruptive MDM vendor AllSight, wrote the blog post The real reason why Master Data Management needs Graph. In here David confirms the common known understanding of that graph databases are superior compared to relational databases when it comes to handle relationships within master data. But David also brings up how graph databases can support multiple versions of the truth.

graph MDMSeveral other vendors as Semarchy and Reltio are emphasizing on graph in MDM in their market messaging.

Aaron Zornes of The MDM Institute is another proponent of using graph technology within MDM as mentioned over at The Disruptive MDM Solutions blog in the post MDM Fact or Fiction: Who Knows?

What do you think: Will graph databases really brake through in MDM soon? Will it be as stand alone graph technology (as for example from neo4j) or embedded in MDM vendor portfolios?

Data Pool vs Data Lake

Henrik Gabs LiliendahlLeave a comment

Within Product Information Management (PIM) – or Product Master Data Management if you like – there is a concept of a data pool.

Recently Justine Rodian of Stibo Systems made a nice blog post with the title Master Data Management Definitions: The Complete A-Z of MDM. Herein Justine explains a lot of terms within Master Data Management (MDM). A data pool is described as this:

“A data pool is a centralized repository of data where trading partners (e.g., retailers, distributors or suppliers) can obtain, maintain and exchange information about products in a standard format. Suppliers can, for instance, upload data to a data pool that cooperating retailers can then receive through their data pool.”

Now, during the last couple of year I have been working on the concept of applying the data lake approach to product information exchange between trading partners. Justine describes a data lake this way:

“A data lake is a place to store your data, usually in its raw form without changing it. The idea of the data lake is to provide a place for the unaltered data in its native format until it’s needed…..” 

Product Data Lake
MacRitchie Reservoir in Singapore

For a provider of product information, typically a manufacturer, the benefit of interacting via a data lake opposite to a data pool is that they do not have to go through standardization before uploading and thus have to shoehorn the data into a specific form and thereby almost certainly leave out important information and being depending on consensus between competing manufacturers.

For a receiver of information, typically a merchant as a retailer and B2B dealer, the benefit of interacting via a data lake opposite to a data pool is that they can request the data in the form they will use to be most competitive and thereby sell more and reduce costs in product information sharing. This will be further accelerated if the merchant uses several data pools.

In Product Data Lake we even combine the best of the two approaches by encompassing data pools in our reservoir concept – to stay in the water body lingo. Here data pools are refreshed with modern data management technology and less rigid incoming and outgoing streams as announced in the post Product Data Lake Version 1.3 is Live.

Seven Flavors of MDM

Henrik Gabs LiliendahlLeave a comment

Master Data Management (MDM) can take many forms. An exciting side of being involved in MDM implementations is that every implementation is a little bit different which also makes room for a lot of different technology options. There is no best MDM solution out there. There are a lot of options where some will be the best fit for a given MDM implementation.

The available solutions also change over the years – typically by spreading to cover more land in the MDM space.

In the following I will shortly introduce the basic stuff with seven flavours of MDM. A given MDM implementation will typically be focused on one of these flavours with some elements of the other flavors and a given piece of technology will have an origin in one of these flavours and in more or less degree encompass some more flavors.

7 flavours

The traditional MDM platform

A traditional MDM solution is a hub for master data aiming at delivering a single source of truth (or trust) for master data within a given organization either enterprise wide or within a portion of an enterprise. The first MDM solutions were aimed at Customer Data Integration (CDI), because having multiple and inconsistent data stores for customer data with varying data quality is a well-known pain point almost everywhere. Besides that, similar pain points exist around vendor data and other party roles, product data, assets, locations and other master data domains and dedicated solutions for that are available.

Product Information Management (PIM)

Special breed of solutions for Product Information Management aimed at having consistent product specifications across the enterprise to be published in multiple sales channels have been around for years and we have seen a continuously integration of the market for such solutions into the traditional MDM space as many of these solutions have morphed into being a kind of MDM solution.

Digital Asset Management (DAM)

Not at least in relation to PIM we have a distinct discipline around handling digital assets as text documents, audio files, video and other rich media data that are different from the structured and granular data we can manage in data models in common database technologies. A post on this blog examines How MDM, PIM and DAM Stick Together.

Big Data Integration

The rise of big data is having a considerable influence on how MDM solutions will look like in the future. You may handle big data directly inside MDM og link to big data outside MDM as told in the post about The Intersection of MDM and Big Data.

Application Data Management (ADM)

Another area where you have to decide where master data stops and handling other data starts is when it comes to transactional data and other forms data handled in dedicated applications as ERP, CRM, PLM (Product Lifecycle Management) and plenty of other industry specific applications. This conundrum was touched in a recent post called MDM vs ADM.

Multi-Domain MDM

Many MDM implementations focus on a single master data domain as customer, vendor or product or you see MDM programs that have a multi-domain vision, overall project management but quite separate tracks for each domain. We have though seen many technology vendors preparing for the multi-domain future either by:

  • Being born in the multi-domain age as for example Semarchy
  • Acquiring the stuff as for example Informatica and IBM
  • Extend from PIM as for example Riversand and Stibo Systems

MDM in the cloud

MDM follows the source applications up into the cloud. New MDM solutions naturally come as a cloud solution. The traditional vendors introduce cloud alternatives to or based on their proven on-promise solutions. There is only one direction here: More and more cloud MDM – also as customer as business partner engagement will take place in the cloud.

Ecosystem wide MDM

Doing MDM enterprise wide is hard enough. But it does not stop there. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus we will have a need for working on the same foundation around master data as reported in the post Ecosystem Wide MDM.

Ecosystem Wide MDM

Henrik Gabs Liliendahl2 Comments

Doing Master Data Management (MDM) enterprise wide is hard enough. The ability to control master data across your organization is essential to enable digitalization initiatives and ensure the competitiveness of your organization in the future.

But it does not stop there. Increasingly every organization will be an integrated part of a business ecosystem where collaboration with business partners will be a part of digitalization and thus we will have a need for working on the same foundation around master data.

The different master data domains will have different roles to play in such endeavors. Party master will be shared in some degree but there are both competitive factors, data protection and privacy factors to be observed as well. However, privacy regulations as GDPR article 20 on data portability will make data sharing a must too.

MDM Ecosystem

Product master data – or product information if you like – is an obvious master data domain where you can gain business benefits from extending master data management to be ecosystem wide. This includes:

  • Working with the same product classifications or being able to continuously map between different classifications used by trading partners
  • Utilizing the same attribute definitions (metadata around products) or being able to continuously map between different attribute taxonomies in use by trading partners
  • Sharing data on product relationships (available accessories, relevant spare parts, updated succession for products, cross-sell information and up-sell opportunities)
  • Having access to latest versions of digital assets (text, audio, video) associated with products

The concept of ecosystem wide Multi-Domain MDM is explored further is the article about Master Data Share.

(PS: Ecosystem wide MDM is coined by Gartner, the analyst firm, as multienterprise MDM and later as Interenterprise MDM).

Spreadsheets, Business Process Re-engineering and Robots

Henrik Gabs LiliendahlLeave a comment

Product information is the data a potential buyer of a product needs to make a purchasing decision. Today purchasing is more and more made by self-services as in e-commerce. The product information is usually obtained through a supply chain between trading partners stretching from the manufacturer to the end merchant.

The most common way of exchanging product information between trading partners is using spreadsheets. Spreadsheets are marvellous, because you can do almost anything you want with them. However, spreadsheets are also horrendous, because you can do almost anything you want with them. Therefore, trading partners are often stuck with manual, cumbersome and error prone processes on both the providing and receiving end.

At Product Data Lake we have developed a new mechanism that enables a whole new process for exchanging product information between trading partners. We have kept the flexibility of spreadsheets when it comes to choosing the data standards on the providing and receiving end but at the same time introduced automation and correctness when it comes to transferring, translating and transforming the data.

When telling about our service I am often asked if we have a nice feature for on-boarding spreadsheets. We don’t. Because the process is designed to omit the spreadsheets and transfer directly from the providers in-house product information data store(s) to the receiving in-house product information data store.

RobotThis reminds me of when we talk about using robots to substitute human labor. Then we often think about a machine that looks like a human. But effective industrial robots do not look like humans. They a designed to do a specific process much more effective than a human and will therefore not look like a human. The same is true in digitalization. When we redesign business processes to be much more effective they should not include spreadsheets.

MDM vs ADM

Henrik Gabs Liliendahl4 Comments

The term Application Data Management (ADM) has recently been circulating in the Master Data Management (MDM) world as touched in The Disruptive MDM List blog post MDM Fact or Fiction: Who Knows?

Not at least Gartner, the analyst firm, has touted this as one of two Disruptive Forces in MDM Land. However, Gartner is not always your friend when it comes to short, crisp and easy digestible definitions and explanations of the terms they promote.

In my mind the two terms MDM and ADM relates as seen below:

ADM MDM.png

So, ADM takes care of a lot of data that we do not usually consider being master data within a given application while MDM takes care of master data across multiple applications.

The big question is how we handle the intersection (and sum of intersections in the IT landscape) when it comes to applying technology.

If you have an IT landscape with a dominant application like for example SAP ECC you are tempted to handle the master data within that application as your master data hub or using a vendor provided tightly integrated tool as for example SAP MDG. For specific master data domains, you might for example regard your CRM application as your customer master data hub. Here MDM and ADM melts into one process and technology platform.

If you have an IT landscape with multiple applications, you should consider implementing a specific MDM platform that receives master data from and provides master data to applications that takes care of all the other data used for specific business objectives. Here MDM and ADM will be in separated processes using best-of-breed technology.

Product Data Completeness

Henrik Gabs LiliendahlLeave a comment

Completeness is one of the most frequently mentioned data quality dimensions as touched in the post How to Improve Completeness of Data.

ChecklistWhile every data quality dimension applies to all domains of Master Data Management (MDM), some different dimensions apply a bit more to one of the domains or the intersections of the domains as explained in the post Multi-Domain MDM and Data Quality Dimensions.

With product master data (or product information if you like) completeness is often a big pain. One reason is that completeness means different requirements for different categories of products as pondered in the post Hierarchical Completeness within Product Information Management.

At Product Data Lake we develop a range of cloud service offerings that will help you improve completeness of product data. These are namely:

  • Measuring completeness against these industry standards that have attribute requirements such as eClass and ETIM
  • For manufacturers measuring completeness against downstream trading partner requirements (if not fully governed by an industry standard).
  • For merchants measuring incoming completeness when pulling from merchants.
  • Measuring against completeness required by marketplaces.
  • Transforming product information to meet conformity and thereby ability to populate according to requirements
  • Translating product information in order to populate attributes in more languages
  • Transferring product information by letting manufacturers push it in their way and letting merchants pull it their way as described in the post Using Pull or Push to Get to the Next Level in Product Information Management.

The Lesser Known GDPR Article 101 (Right to Indulgence)

Henrik Gabs Liliendahl1 Comment

While a lot have been written about the original 99 articles in the EU General Data Protection Regulation (GDPR) perhaps we have seen less focus on the additional articles that have been added since the first approval of 14th April 2016, but that will as well apply on 25th May 2018.

The original articles are mainly emphasizing on the rights of the data subject. But later additions also take care of the right of data controllers. Not at least article 101 about the right to indulgence for data controllers (and data processors) falls into that category.

A representative of the EU, Don Par, explains it this way: “You can escape the heavy fines if you are about to be caught by confessing your sins and pay a more modest purification fee which eventually by subscribing to an annual scheme may allow you to continue business as usual”.

13indulgences.large (1).jpgThe consultancy industry also has had a low profile on article 101. Max Hagnaður of The GDPR Advisory Institute puts it this way: “First we wanted to make money on advising on how to comply with GDPR. When it shows up, that only a very few companies will do so, we will redirect our hailstorms of power-point decks to article 101 and how to apply for indulgence”.

Indeed, the costs of indulgence may very well be lesser than the costs of compliance (not to say fines because you will fail anyway).