The Data Enrichment ABC

A popular and indeed valuable method of avoiding decay of data quality in customer master data and other master data entities is setting up data enrichment services based on third party reference data sources. Examples of such services are:

  • Relocation updates like National Change Of Address services from postal services
  • Change of name, address and a variety of status updates from business directories and in some countries citizen directories too

When using such services you will typically want to consider the following options for how to deal with the updates:

A: Automatic Update

Here your internal master data will be updated automatically when a change is received from the external reference data source.

C: Excluded Update

Here an automated rule will exclude the update as there may be a range reasons for why you don’t want to update certain entity segments under certain circumstances.

B: Interactive Update

Here the update will require a form of manual intervention either to be fulfilled or excluded based on human decision.

An example will be if a utility supplier receives a relocation update for the occupier at an installation address. This will trigger/support a complex business process far beyond changing the billing address.

iDQ logo

As explained in the post When Computer Says Maybe we need functionality within data quality tools and Master Data Management (MDM) solutions to support data stewards in cost effectively handling these situations and this certainly also applies to the B pot in data enrichment.

Right now I’m working with designing such data stewardship functionality within the instant Data Quality environment.

Bookmark and Share

Social Score Credibility

A recent piece from Fliptop is called What’s the Score. It is a thorough walk through on what is usually called social scoring done in influence scoring platforms within social media, where Klout, Kred and PeerIndex are the most known services of that kind.

The Fliptop piece has a section around faking, which was also the subject in a post lately on this blog. The post is called Fact Checking by Mashing Up, and is about how to link social network profiles with other known external sources in order to detect cheat. Linking social network profiles with other external sources and internal sources is what is known as Social MDM, a frequent subject on this blog for several years.

A social score must of course be seen in context, as it matters a lot what you are influential about when you want to use social scoring for business. As told in the post Klout Data Quality this was a challenge two years ago, and this is probably still the case. Also here I think linking with other (big) data sources and letting Social MDM be the hub will help.

Taken from Kred on my twitter handle.

PS: I have no idea why moron ended up there. Einstein is OK.

Bookmark and Share

The Future of Data Stewardship

Data Stewardship is performed by data stewards.

What is a Data Steward?

A steward may in a general sense be:

  • One employed in a large household or estate to manage domestic concerns – typically an old role.
  • An employee on a ship, airplane, bus, or train who attends passengers needs – typically a new role.

My guess is that data stewardship also will tend to be going from the first kind of role related to data to the latter kind role related to data.

The current data steward role is predominately seen as the oversight of the house-holding related to the internal enterprise data assets. It’s about keeping everything there clean and tidy. It involves having routines and rules that ensure that things with data are done properly according to the traditions and culture in the enterprise.

Big Data Stewardship

In the future enterprises will rely much more on external data. Exploiting third party reference data and open government data and digging into big data sources as social data and sensor data will shift the focus from looking mostly into keeping the internal data fit for purposes.

As such you as a data steward will become more like the steward on a ship, airplane, bus or train. Data will come and go. After a nice welcoming smile you will have to carefully explain about the safety procedures. Some data will be fairly easy to handle – mostly just spending the time sleeping. Other data will be demanding asking for this and that and changing its mind shortly after. Some data will be a frequent traveler and some data will be there for the first time.

So, are you ready to attend the next batch of travelling data on board your enterprise?

star trek enterprise

Bookmark and Share

How is Social MDM different?

In a recent interview with yours truly on the Fliptop blog I had the chance to answer a question about how Social MDM is different from traditional MDM (Master Data Management). Check out the interview here.

As said in the interview I think that:

“The main difference between MDM as it has been practiced until now and Social MDM is that traditional MDM has been around handling internal master data and Social MDM will be more around exploiting external reference data and sharing those data.”

This is in line with a take away from the MDM Summit Europe 2013 as reported in the post Adding 180 Degrees to MDM.

But, as asked by a member of the Social MDM group on LinkedIn:

What is the industry or analysts’ consensus on the meaning of Social MDM? Is it just gathering Master Data from social sources? Not really MDM – where is the Management part?”

Social MDM IconYou may follow the discussion here.

I definitely think that the management part is there, but it is different. Management is different in the social sphere in general. Data governance is different when it comes to social data (and other big data for that matter). Relying on social collaboration when maintaining master data is different from implementing “a data steward regime”.

In my eyes the management part is about balancing the use of internal master and the use of external reference data. Every organization should very carefully assess if they are good at maintaining different aspects of their internal master data (Hint: Many aren’t). Getting help from traditional data collectors and the new social sources and using social collaboration may very well be an important part of the solution.

Bookmark and Share

Fact Checking by Mashing Up

A recent blog post by Andrew Grill, CEO of Kred, is called Can you spot a social media faker? Fact checking on social media is now becoming even more important.

Besides methods within the social sphere for fact checking, as described in Andrew Grill’s post, I also believe that mashing up social network profiles and traditional external reference data is a great way of getting the full picture.

As explained in the post Sharing is the Future of MDM there are several available external options for checking the facts:

  • Public sector registries which are getting more and more open being that for example for the address part or even deeper in due respect of privacy considerations which may be different for business entities and individual entities.
  • Commercial directories often build on top of public registries.
  • Personal data lockers like Mydex
  • Social network profiles, including credibility (or influence) services

The challenge is of course that there are plenty of external reference data sources as many sources are national, making up 255 or so variants of each data source, as well as there are plenty of social networks and some credibility (or influence) services for that matter.

Making that easy for you is exactly the concept we are working on in the instant Data Quality, iDQ™, concept.


Bookmark and Share

Adding 180 Degrees to MDM

Master Data Management (MDM) has traditionally been about being better at utilizing and sharing internal registrations about our customers, suppliers, products, assets and other core business entities.

My latest work around master data management revolves around the concept of bringing in external data sources in order to make on-boarding processes more efficient and provide more accurate, complete and timely master data.

So, it was good to see that this approach is gaining more traction when attending the MDM Summit Europe 2013.

The old stuff

Andy Walker of BP presented how BP has built the management of party master data around aligning with the D&B WorldBase for business-to-business (B2B) customer and vendor master data.

Knowing about with which actual legal entities you are doing business and which external hierarchies they belong to is crucial for BP both in daily operations and when it comes to reporting and analysis utilizing party master data.

Using business directories isn’t new at all; it has been around for ages and from what I have seen: It works when you do it properly and consistently.

The new stuff

Big data was a hot topic on the conference. As reported in a post from the first day embracing big data may lead to Double Trouble with Social MDM and Big Data.

Steve Jones TweetHowever, digging into big data and doing social MDM may certainly also provide new opportunities as we by utilizing these new sources actually may be able to obtain (or closing in at) a 360 degree view on various master data entity types. It is, as said and tweeted by Steve Jones of Capgemini, about looking outside-in.

Bookmark and Share

Sharing is the Future of MDM

Over at the DataRoundtable blog Dylan Jones recently posted an excellent piece called The Future of MDM?

Herein Dylan examines how a lot of people in different organizations spend a lot of time on trying to get complete, timely and unique data about customers and other business partners.

A better future for MDM (Master Data Management) could certainly be that every organization doesn’t have to do the work over and over and again. While self registration by customers is a way of letting off the burden on private enterprises and public sector bodies, we may even do better by not having the customer being the data entry clerk and typing in the same information over and over and again.

Today there are several available options for customer and other business partner reference data:

  • Public sector registries which are getting more and more open being that for example for the address part or even deeper in due respect of privacy considerations which may be different for business entities and individual entities.
  • Commercial directories often build on top of public registries.
  • Personal data lockers like the Mydex service mentioned by Dylan.
  • Social network profiles.

instant Single Customer ViewMy guess is that the future of MDM is going to be a mashup of exploiting the above options.

Oh, and as representatives of such a mashup service we recently at iDQ made sure we had the accurate, complete and timely information filled in on our Linkedin Company profile.

Bookmark and Share

Data Management in the Cloud

We are seeing more and more data management services offered in the cloud.

dnblogo2As I have had a long time experience with data matching services around the Dun & Bradstreet WorldBase, it was good to see a presentation yesterday in Stockholm featuring D&B Europe’s new cloud based data manager service.

Managing World-Wide B2B Master Data

The D&B WorldBase is a business directory with 225 million business entities from all over the world.

D&B’s Data Manager is a self-service application in the cloud around the WorldBase taking care of:

  • Data matching with comprehensive functionality for manual inspection, approval and master data survivorship
  • Data enrichment embracing a wide range of data attributes
  • Data Maintenance subscription for keeping enriched data up to date

The data matching functionality is built on the good old D&B methodology with confidence codes and matchgrades.

Right for QlikTech

QlikTech is the Swedish firm (pretending to be American) behind the prominent business intelligence solution called QlikView.

At the Stockholm event QlikTech presented how and why they use the D&B Data Manager for ensuring the right data quality in their cloud based B2B CRM solution (

As QlikTech is operating all over the world having a consistent world-wide business directory as the reference for party master data is extremely important, and the self-service concept is a perfect match for having the right insight and control into achieving the needed level of data quality in CRM master data.

From there the QlikTech CRM team takes its own medicine using QlikView for self-service business intelligence.

Bookmark and Share

instant Single Customer View

Achieving a Single Customer View (SCV) is a core driver for many data quality improvement and Master Data Management (MDM) implementations.

As most data quality practitioners will agree, the best way of securing data quality is getting it right the first time. The same is true about achieving a Single Customer View. Get it right the first time. Have an instant Single Customer View.

The cloud based solution I’m working with right now does this by:

  • Searching external big reference data sources with information about individuals, companies, locations and properties as well as social networks
  • Searching internal master data with information already known inside the enterprise
  • Inserting really new entities or updating current entities by picking  as much data as possible from external sources

instant Single Customer View

Some essential capabilities in doing this are:

  • Searching is error tolerant so you will find entities even if the spelling is different
  • The receiving data model is real world aligned. This includes:
    • Party information and location information have separate lives as explained in the post called A Place in Time
    • You may have multiple means of contact attached like many phones, email addresses and social identities

How do you achieve a Single Customer View?

Bookmark and Share

Master Data Management in the Utility Sector

Making vertical MDM (Master Data Management) solutions, being MDM solutions prepared for a given industry, seems to become a trend in the MDM realm.

Traditionally many MDM solutions actually are strong in a given industry or a few related industries.

This is also true for the MDM solution I’m working with right now, as this solution has gained traction in the utility sector.

So, what’s special (and not entirely special) about the utility sector?

Here are three of my observations:

Exploiting big external reference data

As examined in the post instant Data Quality at Work the utility sector may gain much in using all the available external reference data available in the party master data domain, including:

  • Consumer/citizen directories
  • Business directories
  • Address directories
  • Property directories

However, if data quality shouldn’t be a joke, this means using the best national data sources available as many of the world-wide data sources is this domain are far from providing the precision, accuracy and timeliness needed in the utility sector.

Location precision

Managing locations is a big thing in the utility sector. The post called Where is the Spot explains how identifying locations isn’t as simple as we may use to think in daily life.

This is indeed also true in the utility sector where the issue also includes managing many different locations for the same customer fulfilling different purposes at the same time.

The products

puzzleThe electricity supply part of the utility sector share a lot of issues with the telco sector when it comes to fixed installations and the products and services are in fact the same in some cases which also as a consequence means that  some organizations belongs to both sectors.

This is also a danger with vertical MDM solutions as there may be several best-of-breed options for a given organization, which eventually will result in choosing more than one platform and thereby introducing the silos which MDM in first place was supposed to eliminate.