As usual hopes are high for the England soccer team. But statistics doesn’t support the hopes. The England team haven’t really succeeded since the World Cup victory on home ground at Wembley in 1966. That victory was mainly (and now I’m going to be shot in the streets of London) due to a ghost goal.

In business, and in data quality and MDM business too, the home team usually also wins.

Yesterday I noticed a tweet telling that the MDM tool vendor Orchestra Network has been selected as tool vendor by a large bank. The bank is Credit Agricole, a big financial service provider based in France. Orchestra Networks is also based in France. A home win so to say.

In the post The Pond it was told how else dominating American tool vendors may in the first place succeed in expansion to Europe by coming to London, but in fact having a hard time competing in continental Europe due to diversity issues.

European tool vendors going to North America often tries to disguise as a home team. Orchestra Network for example uses Boston & Paris as place of origin in the messaging. Other examples are the leading open source data management tool vendor Talend with dual head quarter in Paris and California, hot Danish MDM vendor Stibo Systems messaging out of Atlanta and the Swedish business intelligence success QlikTech who officially has moved to Pennsylvania.

I agree.

The problem with the “fitness for use” or “fit for the purpose of use” definition of data quality has been a recurring subject on this blog starting with the post Fit for What Purpose? through to lately the post Inaccurately Accurate discussing the data quality of the British electoral roll seen from either a strict electoral point of view and the point of view from external use of the electoral roll.

The problem with “fitness of use” becomes clear when data quality has to be addressed within master data management. Master data has, per definition so to say, many uses.

My thesis is that there is a breakeven point when including more and more purposes where it will be less cumbersome to reflect the real world object rather than trying to align all known purposes.

Today Jim Harris made an (as ever) excellent post related to how data actually represents what it purports to represent – now and tomorrow too. Find the post called Syncing versus Streaming on the Data Roundtable.

There is clearly something completely wrong with this tweet. Why on earth should a French company use an American date format?

Apart from that there is a very good point. Why should tool vendors work on solving imaginable future master data management issues as integrating social network profiles with traditional customer master data while there are plenty of issues that need a better solution today?

Personally I think social MDM is going to be huge. I had some of my first musings on the subject some years ago in the post Social Master Data Management. Probably we will start with some Lean Social MDM, and that is honestly also as far as I have explored this field until now.

What about you. Do you want social MDM?

Cleansing data long time after it has been captured is a common way of dealing with this huge problem. However, preventing typos, wrong hearings and multi-cultural misunderstandings at data entry is a much better option wherever applicable.

I have worked with two different approaches to ensure the best data quality for contact data entered by employees. These approaches are:

• Correction and
• Assistance

Correction

With correction the data entry clerk, sales representative, customer service professional or whoever is entering the data will enter the name, address and other data into a form.

After submitting the form, or in some cases leaving each field on the form, the application will check the content against business rules and available reference data and return a warning or error message and perhaps a correction to the entered data.

As duplicated data is a very common data quality issue in contact data, a frequent example of such a prompt is a warning about that a similar contact record already exists in the system.

Assistance

With assistance we try to minimize the needed number of key strokes and interactively help with searching in available reference data.

For example when entering address data assistance based data entry will start with the highest geographical level:

• If we are dealing with international data the country will set the context and know about if a state or province is needed.
• Where postal codes (like ZIP) exists, this is the fast path to the city.
• In some countries the postal code only covers one street (thoroughfare), so that’s settled by the postal code. In other situations we will usually have a limited number of streets that can be picked from a list or settled with the first characters.

(I guess many people know this approach from navigation devices for cars.)

When the valid address is known you may catch companies from business directories being on that address and, depending on the country in question, you may know citizens living there from phone directories and other sources and of course the internal party master data, thus avoiding entering what is already known about names and other data.

When catching business entities a search for a name in a business directory often leads to being able to pick a range of identification data and other valuable data and not at least a reference key to future data updates.

Lately I have worked intensively with an assistance based cloud service for business processes embracing contact data entry. We have some great testimonials about the advantages of such an approach here: instant Data Quality Testimonials.

In a blog post called Informatica 9.5 for Big Data Challenge #2: Social Jody Ko of Informatica introduces the opportunities and challenges.

In the closing remarks Judy says: “There’s still a long way to go to bring social data into the mainstream enterprise, in part due to concerns over privacy and the potential “creepiness” factor of mining social data.”

As I understand it the spearhead Social MDM part of the tool release is a Facebook App that provides connectivity between Facebook and the MDM solution.

Industry analyst R “Ray” Wang examines this in the blog post News Analysis: Informatica Launches MDM 9.5. The analysis states that it now is time to “drive data out of Facebook and not into Facebook”.

The opportunities and challenges of driving data out of Facebook was discussed in a post called exactly Out of Facebook here on the blog some years ago.

Balancing privacy with data hoarding is still for sure a subject that in no way is settled and probably never will be.

Connecting systems of record in traditional MDM solutions with social network profiles is in no way a walk over too. The classic data quality challenges with uniqueness of records and completeness of data only gets more difficult, but also, there are great opportunities for getting a better picture of your customers and other business partners.

If you are interested in Social MDM and the related challenges and opportunities there is a LinkedIn group for Social MDM.

The group is new, less than a month old at the present time, but there is already a lot of content to dip into, including:

• A discussion called: What’s the business case?
• A survey on data quality mechanisms focused on author identity resolution in social media
• A poll: What is a good MDMish name for social network profiles?

Traditional data matching as seen in most data quality tools and master data management solutions is the bright view: Being about finding duplicates and making a “single customer view”. Identity resolution is the dark view: Preventing fraud and catching criminals, terrorists and other villains.

These two poles were discussed in a blog post and the following comments last year. The post was called What is Identity Resolution?

While deduplication and identity resolution may be treated as polar opposites and seemingly contrary disciplines they are in my eyes interconnected and interdependent. Yin and Yang Data Quality.

At the MDM Summit in London last month one session was about the Golden Nominal, Creating a Single Record View. Here Corinne Brazier, Force Records Manager at the West Midlands Police in the UK told about how a traditional data quality tool with some matching capabilities was used to deal with “customers” who don’t want to be recognized.

In the post How to Avoid Losing 5 Billion Euros it was examined how both traditional data matching tools and identity screening services can be used to prevent and discover fraudulent behavior.

Deduplication becomes better when some element of identity resolution is added to the process. That includes embracing big reference data in the process. Knowing what is known in available sources about the addresses that is being matched helps. Knowing what is known in business directories about companies helps. Knowing what is known in appropriate citizen directories when deduping records holding data about individuals helps.

Identity Resolution techniques is based on the same data matching algorithms we use for deduplication. Here for example a fuzzy search technology helps a lot compared to using wildcards. And of course the same sources as mentioned above are a key to the resolution.

Right now I’m dipping deep into the world of big reference data as address directories, business directories, citizen directories and the next big thing being social network profiles. I have no doubt about that deduplication and identity resolution will be more yinyang than yin and yang in the future.

Häagen-Dazs ice cream is an example of foreign branding. Though the brand was established in New York the name was supposed to sound Scandinavian.

However, Häagen-Dazs does sound and look somewhat strange to a Scandinavian. The reason is probably that the constellation of the letters “äa” and “zs” are not part of any native Scandinavian words.

By the way, datakvalitet is the Scandinavian compound word for data quality.

Getting datakvalitet right in world wide data isn’t easy. What works in some countries doesn’t work in other countries, not at least when we are talking datakvalitet regarding party master data such as customer master data, supplier master data and employee master data.

One of the reasons why datakvalitet for party master data is different is the various possibilities with applying big reference data sources. For example the availability of citizen data is different in New York than in Scandinavia. This affects the ways of reaching optimal datakvalitet as reported in the post Did They Put a Man on the Moon.

As part of the ongoing globalization handling international datakvalitet is becoming more and more common. Many enterprises try to deploy enterprise wide datakvalitet initiatives and shared service centers handles party master data uncommon to the people working there. This often results in finding a strange word like Häagen-Dazs.

Gartner has its magic quadrant. They sell it for money, but usually you are able to get a free copy from the leading vendors.

The Information Difference has its DQ Landscape in the cloud for free.

It is interesting to compare which vendors are included in the latest main pictures, as I have tried below:

The number of x’s is a rough measure of the ability to execute / market strength.

Three smaller vendors are considered by Gartner, but not by The Information Difference and vice versa. Two midsize vendors are included by The Information Difference, but not by Gartner. Experian QAS are included as a big one by The Information Difference, but did not (yet) meet the inclusion criteria used by Gartner.

In complex sales processes, typically for Business-to-Business (B2B) sales, the website may offer product information sheets, demo requests, contact forms and other pipeline steps.

This is the moment where your social media engaged (prospective) customer meets your master data as:

• The (prospective) customer creates and maintains name, address and communication information by using registration functions
• The (prospective) customer searches for and reads product information on web shops and information sites

One aspect of this transition is how master data is carried over, namely:

• How the social network profile used in engagement is captured as part of (prospective) customer master data or if it should be part of master data at all?
• How product information from the governed master data hub has been used as part of the social media engagement or if the data governance of product data should be extended to use in social media at all?

Any thoughts?

In the carbon trade scam criminals were able to trick authorities with fraudulent names and addresses.

One way of possible discovery of the fraudster’s pattern of interrelated names and physical and digital locations was, as explained in the post, to have used an “off the shelf” data matching tool in order to achieve what is sometimes called non-obvious relationship awareness. When examining the data I used the Omikron Data Quality Center.

Another and more proactive way would have been upstream prevention by screening identity at data capture.

Identity checking may be a lot of work you don’t want to include in business processes with high volume of master data capture, and not at least screening the identity of companies and individuals on foreign addresses seems a daunting task.

One way to help with overcoming the time used on identity screening covering many countries is using a service that embraces many data sources from many countries at the same time. A core technology in doing so is cloud service brokerage. Here your IT department only has to deal with one interface opposite to having to find, test and maintain hundreds of different cloud services for getting the right data available in business processes.

Right now I’m working with such a solution called instant Data Quality (iDQ).

Really hope there’s more organisations and organizations out there wanting to avoid losing 5 billion Euros, Pounds, Dollars, Rupees, Whatever or even a little bit less.