IoT and Multi-Domain MDM

The Internet-of-Things (IoT) is a hot topic and many Master Data Management (MDM) practitioners as well as tool and service vendors are exploring what the rise of the Internet-of-Things and the related Industry 4.0 themes will mean for Master Data Management in the years to come.

globalIn my eyes, connecting these smart devices and exploiting the big data you can pull (or being pushed) from them will require a lot for all Master Data Management domains. Some main considerations will be:

  • Party Master Data Management is needed to know about the many roles you can apply to a given device. Who is the manufacturer, vendor, supplier, owner, maintainer and collector of data? Privacy and security matters on that basis will have to be taken very seriously.
  • Location Master Data Management is necessary at a much deeper and precise level than what we are used to when dealing with postal addresses. You will need to know a home location with a timespan and you will need to confirm and, for moving devices, supplement with observed locations with a timestamp.
  • Product and Asset Master Data Management is imperative in order to know about the product model of the smart device and individual characteristics of the given device.

It is also interesting to consider, if you will be able to manage this connectivity within a MDM platform (even multidomain and end-to-end) behind your corporate walls. I do not think so as told in the post The Intersections of 360 Degree MDM.

The Pros and Cons of Master Data (Management)

As a comment on my LinkedIn status about my previous post Jan van Til asks:

Wouldn’t the world be far better of without the concept of master data? What problems are solved by it? What problems are introduced by it? The balance? So … why do we keep toiling with master data?”

What problems are solved?

A common definition of data quality is that data are fit for the intended purpose of use. However, with master data we have multiple purposes of use of these core data entities. In an enterprise architecture with no focus on master data, the same real world construct will be described in many different databases in many different ways.

This causes a myriad of challenges. There are no one face to our customers, which makes us look stupid. We may offer the same product unintentionally with different prices and with different features, which is stupid. Our reporting, business intelligence, data science will be based on ambiguous data and therefore potentially cause stupid decisions.

If we have data quality issues with no centralized management of master data we will fund data cleansing and prevention initiatives many times for the same real world construct – and probably with varying outcome. In a centralized master data management set up, we can avoid reinventing the wheel as explained in the post The Database versus the Hub.

What problems are introduced?

Implementing Master Data Management (MDM) is not without tears. This involves the whole enterprise, and will increasingly caused by the rise of digitalization involve the business ecosystem as well. You will have to spend money and resources, which are not always easily justified.

MDM is a complex discipline involving many stakeholders. There is a high risk of running over budget and time and missing the goals.

As MDM is the remedy against data silos, you may end up with MDM as just another data silo within your enterprise. And still, your MDM hub may be a data silo in your business ecosystem.

The balance

Think big, start small. Make it agile. A good example of an agile approach to MDM is proposed by the MDM vendor Semarchy as described here on What is Evolutionary MDM™?

While you need a strong inner hub for master data in your enterprise, do not try to impose that hub on everything and everyone as for example examined in the post A Different End-to-End Solution for Product Information Management (PIM).

Multi-Side MDM

What is a Master Data Entity?

What is a customer? What is a product? You encounter these common questions when working with Master Data Management (MDM).

The overall question about what master data is has been discussed on this blog often as for example in the post A Master Data Mind Map.

Master Data

The two common questions posed as start of this blog post is said to be very dangerous. Well, here are my experiences and opinions:

What is a customer?

In my eyes, customer is a role you can assign to a party. Therefore, the party is the real master data entity. A party can have many other roles as employee, supplier and other kinds of business partner roles. More times than you usually imagine, the party can have several roles at the same time. Examples are customers also being employees and suppliers who are also customers.

From a data quality point of view, it does not have to matter if a party is a customer or not at a certain time. If your business rules requires you to register that party because the party has placed an order, got an invoice, paid an invoice or pre-paid an amount, you will need to take care of the quality of the information you have stored. You will also have to care about the privacy, not at least if the party is a natural person.

Uniqueness is the most frequent data quality issue when it comes to party master data. Again, it is essential to detect or better prevent if the same party is registered twice or more whether that party is a customer according to someone’s definition or not.

What is a product?

Also with products business rules dictates if you are going to register that product. If you are a reseller of products, you should register a product that you promote (being in your range). You could register a product, that you resell that product occasionally (sometimes called specials). If you are a manufacturer, you should register your finished products, your semi-finished products and the used raw materials. Most companies are actually both a reseller and a manufacturer in some degree. Despite of that degree practically all companies also deals with indirect goods as spare parts, office supplies and other stuff you could register as a product within your organisation in the same way your supplier probably have.

What we usually defines as a product is most often what rather should be called a product model. That means we register information about things that are made in the same way and up by the same ingredients and branded similarly. A thing, as each physical instance of a product model, will increasingly have business rules that requires it to be registered as told in the post Adding Things to Product Data Lake.

Big Data Fitness

A man with one watch knows what time it is, but a man with two watches is never quite sure. This old saying could be modernized to, that a person with one smart device knows the truth, but a person with two smart devices is never quite sure.

An example from my own life is measuring my daily steps in order to motivate me to be more fit. Currently I have two data streams coming in. One is managed by the app Google Fit and one is managed by the app S Health (from Samsung).

This morning a same time shot looked like this:

Google Fit:

google-fit

S Health:

s-health

So, how many steps did I take this morning? 2,047 or 2413?

The steps are presented on the same device. A smartphone. They are though measured on two different devices. Google Fit data are measured on the smartphone itself while S Health data are measured on a connected smartwatch. Therefore, I might not be wearing these devices in the exact same way. For example, I am the kind of Luddite that do not bring the phone to the loo.

With the rise of the Internet of Things (IoT) and the expected intensive use of the big data streams coming from all kinds of smart devices, we will face heaps of similar cases, where we have two or more sets of data telling the same story in a different way.

A key to utilize these data in the best fit way is to understand from what and where these data comes. Knowing that is achieved through modern Master Data Management (MDM).

At Product Data Lake we in all humbleness are supporting that by sharing data about the product models for smart devices and in the future by sharing data about each device as told in the post Adding Things to Product Data Lake.

Golden Records in Multi-Domain MDM

The term golden record is a core concept within Master Data Management (MDM). A golden record is a representation of a real world entity that may be compiled from multiple different representations of that entity in a single or in multiple different databases within the enterprise system landscape.

GoldIn Multi-domain MDM we work with a range of different entity types as party (with customer, supplier, employee and other roles), location, product and asset. The golden record concept applies to all of these entity types, but in slightly different ways.

Party Golden Records

Having a golden record that facilitates a single view of customer is probably the most known example of using the golden record concept. Managing customer records and dealing with duplicates of those is the most frequent data quality issue around.

If you are not able to prevent duplicate records from entering your MDM world, which is the best approach, then you have to apply data matching capabilities. When identifying a duplicate you must be able to intelligently merge any conflicting views into a golden record.

In lesser degree we see the same challenges in getting a single view of suppliers and, which is one of my favourite subjects, you ultimately will want to have a single view on any business partner, also where the same real world entity have both customer, supplier and other roles to your organization.

Location Golden Records

Having the same location only represented once in a golden record and applying any party, product and asset record, and ultimately golden record, to that record may be seen as quite academic. Nevertheless, striving for that concept will solve many data quality conundrums.

GoldLocation management have different meanings and importance for different industries. One example is that a brewery makes business with the legal entity (party) that owns a bar, café, restaurant. However, even though the owner of that place changes, which happens a lot, the brewery is still interested in being the brand served at that place. Also, the brewery wants to keep records of logistics around that place and the historic volumes delivered to that place. Utility and insurance is other examples of industries where the location golden record (should) matter a lot.

Knowing the properties of a location also supports the party deduplication process. For example, if you have two records with the name “John Smith” on the same address, the probability of that being the same real world entity is dependent on whether that location is a single-family house or a nursing home.

Product Golden Record

Product Information Management (PIM) solutions became popular with the raise of multi-channel where having the same representation of a product in offline and online channels is essential. The self-service approach in online sales also drew the requirements of managing a lot more product attributes than seen before, which again points to a solution of handling the product entity centralized.

In large organizations that have many business units around the world you struggle with having a local view and a global view of products. A given product may be a finished product to one unit but a raw material to another unit. Even a global SAP rollout will usually not clarify this – rather the contrary.

GoldWhile third party reference data helps a lot with handling golden records for party and location, this is lesser the case for product master data. Classification systems and data pools do exist, but will certainly not take you all the way. With product master data we must, in my eyes, rely more on second party master data meaning sharing product master data within the business ecosystems where you are present.

Asset (or Thing) Golden Records

In asset master data management you also have different purposes where having a single view of a real world asset helps a lot. There are namely financial purposes and logistic purposes that have to aligned, but also a lot of others purposes depending on the industry and the type of asset.

With the raise of the Internet of Things (IoT) we will have to manage a lot more assets (or things) than we usually have considered. When a thing (a machine, a vehicle, an appliance) becomes intelligent and now produces big data, master data management and indeed multi-domain master data management becomes imperative.

You will want to know a lot about the product model of the thing in order to make sense of the produced big data. For that, you need the product (model) golden record. You will want to have deep knowledge of the location in time of the thing. You cannot do that without the location golden records. You will want to know the different party roles in time related to the thing. The owner, the operator, the maintainer. If you want to avoid chaos, you need party golden records.

PIM Supplier Portals: Are They Good or Bad?

A recent discussion on the LinkedIn Multi-Domain MDM group is about vendor / supplier portals as a part of Product Information Management implementations.

A supplier portal (or vendor portal if you like) is usually an extension to a Product Information Management (PIM) solution. The idea is that the suppliers of products, and thus providers of product information, to you as a downstream participant (distributor or retailer) in a supply chain, can upload their product information into your PIM solution and thus relieving you of doing that. This process usually replace the work of receiving spreadsheets from suppliers in the many situations where data pools are not relevant.

In my opinion and experience, this is a flawed concept, because it is hostile to the supplier. The supplier will have hundreds of downstream receivers of products and thus product information. If all of them introduced their own supplier portal, they will have to learn and maintain hundreds of them. Only if you are bigger than your supplier is and is a substantial part of their business, they will go with you.

Broken data supply chainAnother concept, which is the opposite, is also emerging. This is manufacturers and upstream distributors establishing PIM customer portals, where suppliers can fetch product information. This concept is in my eyes flawed exactly the opposite way.

And then let us imagine that every provider of product information had their PIM customer portal and every receiver had their PIM supplier portal. Then no data would flow at all.

What is your opinion and experience?

What Will you Complicate in the Year of the Rooster?

rooster-6Today is the first day in the new year. The year of the rooster according to the Lunar Calendar observed in East Asia. One of the characteristics of the year of the rooster is that in this year, people will tend to complicate things.

People usually likes to keep things simple. The KISS principle – Keep It Simple, Stupid – has many fans. But not me. Not that I do not like to keep things simple. I do. But only as simple as it should be as Einstein probably said. Sometimes KISS is the shortcut to getting it all wrong.

When working with data quality I have come across the three below examples of striking the right balance in making things a bit complicated and not too simple:

Deduplication

One of the most frequent data quality issues around is duplicates in party master data. Customer, supplier, patient, citizen, member and many other roles of legal entities and natural persons, where the real world entity are described more than once with different values in our databases.

In solving this challenge, we can use methods as match codes and edit distance to detect duplicates. However, these methods, often called deterministic, are far too simple to really automate the remedy. We can also use advanced probabilistic methods. These methods are better, but have the downside that the matching done is hard to explain, repeat and reuse in other contexts.

My best experience is to use something in between these approaches. Not too simple and not too overcomplicated.

Address verification

You can make a good algorithm to perform verification of postal and visit addresses in a database for addresses coming from one country. However, if you try the same algorithm on addresses from another country, it often fails miserably.

Making an algorithm for addresses from all over the world will be very complicated. I have not seen one yet, that works.

My best experience is to accept the complication of having almost as many algorithms as there are countries on this planet.

Product classification

Classifications of products controls a lot of the data quality dimensions related to product master data. The most prominent example is completeness of product information. Whether you have complete product information is dependent on the classification of the product. Some attributes will be mandatory for one product but make no sense at all to another product by a different classification.

If your product classification is too simple, your completeness measurement will not be realistic. A too granular or other way complicated classification system is very hard to maintain and will probably seem as an overkill for many purposes of product master data management.

My best experience is that you have to maintain several classification systems and have a linking between them, both inside your organization and between your trading partners.

Happy New Lunar Year