Data Architecture

My View

Henrik Gabs Liliendahl7 Comments

This post is inspired by the view from our roof terrace, where I’m sitting with the laptop right now.

One of the buildings I can see in the skyline is the spectacular new Hotel Bella Sky that will open tonight.

The new hotel is situated by the main fair in Copenhagen called Bella Center, the venue of the recent disastrous climate change summit where Wen, Obama and Singh couldn’t agree about anything.     

The Bella Sky isn’t the only new high rising hotel in the nearby skyline. Actually there is currently an overcapacity of hotel rooms in Copenhagen. But as it is said, the new hotels were planned before the credit crunch and couldn’t be stopped.   

Planning several years in advance has always been difficult. Within information technology it’s also a well known fact that projects that is set to deliver some years ahead almost always fails to meet the actual business needs when that time is reached.

On the one hand we need some more agile hotel projects – and agile information technology projects – including agile master data management and data quality programs.

On the under hand, I like it when I see some nice hotel architecture and some good data architecture.

Bookmark and Share

Georgian Geography and History

Henrik Gabs LiliendahlLeave a comment

This is the sixth post in a series of short blog posts focusing on data quality related to different countries around the world. I am not aiming at presenting a single version of the full truth but rather presenting a few random observations that I hope someone living in or with knowledge about the country are able to clarify in a comment.

Georgia

Georgia is the English name for a sovereign state in the South Caucasus where Europe meets Asia. Georgia was a part of the Soviet Union under the English name Georgian SSR from 1922 to 1991. Back in the 4th century BC a unified kingdom of Georgia was established as an early example of an advanced state organization under one king and an aristocratic hierarchy.

Georgia

Georgia is a state located in the southeastern United States. Back in the 18th century the area was known as the Province of Georgia within the British colonies. Before the arrival of the Europeans some of current Georgia was part of the Cofitachequi paramount chiefdom.

Ambiguous place names and slowly changing dimensions

Like with Georgia there are lots of examples of place names belonging to more than one place on Earth. Besides that location reference data like the Georgia’s have slowly changing dimensions as what area is covered, where in a hierarchy it belongs and what it is called at a certain time.

Previous Data Quality World Tour blog posts:

Does One Size Fit Anyone?

Henrik Gabs Liliendahl2 Comments

Following up on a recent post about data silos I have been thinking (and remembering) a bit about the idea that one company can have all master data stored in a single master data hub.

Supply Chain Musings

If you for example look at a manufacturer the procurement of raw materials is of course an important business process.

Besides purchasing raw materials the manufacturer also buys machinery, spare parts for the machinery and maintenance services for the machinery.

Like everyone else the manufacturer also buys office supplies – including rare stuff as data quality tools and master data management consultancy.

If you look at the vendor table in such a company the number of “supporting suppliers” are much higher than the number of the essential suppliers of raw materials. The business processes, data structures and data quality metrics for on-boarding and maintaining supplier data and product data are “same same but very different” for these groups of suppliers and the product data involved.

Supply Chain Centric Selling

I remember at one client in manufacturing a bi-function in procurement was selling bi-products from the production to a completely different audience than the customers for the finished products. They had a wonderful multi-domain data silo for that.

Hierarchical Customer Relations

A manufacturer may have a golden business rule saying that all sales of finished products go through channel partners. That will typically mean a modest number of customers in the basic definition being someone who pays you. Here you typically need a complex data structure and advanced workflows for business-to-business (B2B) customer relationship management.

Your channel partners will then have customers being either consumers (B2B2C) or business users within a wider range of companies. I have noticed an increasing interest in keeping some kind of track of the interaction with end users of your products, and I guess embracing social media will only add to that trend. The business processes, data structures and data quality metrics for doing that are “same same but very different” from your basic customer relationship management.

Conclusion

The above musings are revolved around manufacturing companies, but I have met similar ranges of primary and secondary constructs related to master data management in all other industry verticals.   

So, can all master data in a given company be handled in a single master data hub?

I think it’s possible, but it has to be an extremely flexible hub either having a lot of different built-in functionality or being open for integration with external services.

Bookmark and Share

Boiling Data Silos

Henrik Gabs Liliendahl6 Comments

Yesterday there where some blog posts dealing with data silos.

Graham Rhind posted: Data silos – learn to live with them.

Rob Karel posted: Stop trying to put a monetary value on data – it’s the wrong path. Though not being the main subject there was a remark saying: “Attempting to boil the ocean and trying to solve Customer, Product, or Financial data for all processes and decisions across the whole organization is too big an effort destined to fail before it starts”.  

Mark Montgomery made a comment on Rob’s post saying: “I also have trouble with the boil the ocean metaphor, which is used too often these days to justify all kinds of protectionist policies in the enterprise. You can’t have it both ways in the enterprise– either you have data silos or you don’t, and I argue that increasingly the world cannot afford them, albeit in highly secure formats in most situations”.

I guess we have to go for the golden mean on this one also. We shouldn’t accept data silos but we must expect them. We could go for eliminating them probably not in one big bang but slice by slice as we climb up the levels in an information maturity model.

I would definitely expect to see fewer and smaller data silos at the top level of an information maturity model than on a bottom level of a data quality immaturity model.

Bookmark and Share

Holistic Accuracy

Henrik Gabs LiliendahlLeave a comment

In community economics you have two terms called

  • Partitive accuracy and
  • Holistic accuracy

In short, partitive accuracy is the accuracy of a single measure being part of a model while holistic accuracy is the accuracy of the model structure and its use. More information here.

I find these terms being very useful in data quality and master data management as well.

The distinction between partitive accuracy and holistic accuracy resembles the distinction between data quality and information quality.

One problem with the term information quality is that it implies a certain context of use, which makes it hard to prepare data for having high data quality for multiple uses other than assuring the accuracy of the single data elements – being similar to the term partitive accuracy.

One clue for assuring better information quality is looking at the model structure of data – being similar to the term holistic accuracy. Here I am thinking beyond traditional data modeling, which is anchored in the technical world, and into how end users of master data hubs are able to build structures of data (with partitive accuracy) that fits the daily business use.

Examples of such holistic information capabilities in master data management will be building flexible product hierarchies and hierarchies of party master data that at the same time reflects hierarchies in the real world as households and company family trees and hierarchies of related accounts and addresses used within the enterprise.

While a single data element as an address component like a postal code may be partitive accurate, the holistic accuracy is seen as how data elements contribute to a holistic accuracy as a part of a data structure that fits multiple purposes of use.

Bookmark and Share

Non-Obvious Entity Relationship Awareness

Henrik Gabs Liliendahl2 Comments

In a recent post here on this blog it was discussed: What is Identity Resolution?

One angle was the interchangeable use of the terms “Identity Resolution” and “Entity Resolution”. These terms can be seen as truly interchangeable, as that “Identity Resolution” is more advanced than “Entity Resolution” or as (my suggestion) that “Identity Resolution” is merely related to party master data, but “Entity Resolution” can be about all master data domains as parties, locations and products.

Another term sometimes used in this realm is “Non-Obvious Relationship Awareness”. Also this term is merely related to finding relationships between parties, for example individuals at a casino that seems to do better than the croupiers. Here’s a link to a (rather old) O’Reilly Radar post on Non-Obvious Relationship Awareness.

Going Multi-Domain

So “Non-Obvious Entity Relationship Awareness” could be about finding these hidden relationships in a multi-domain master data scope.

An example could be non-obvious relationships in a customer/product matrix.

The data supporting this discovery will actually not be found in the master data itself, but in transaction data probably being in an Enterprise Data Warehouse (EDW). But a multi-domain master data management platform will be needed to support the complex hierarchies and categorizations needed to make the discovery.   

One technical aspect of discovering such non-obvious relationships is how chains of keys are stored in the multi-domain master data hub.

Customer Master Data

The transactions or sums hereof in the data warehouse will have keys referencing customer accounts. These accounts can be stored in staging areas in the master data hub with references to a golden record for each individual or company in the real world. Depending on the identity resolution available the golden records will have golden relations to each other as they are forming hierarchies of households, company family trees, contacts within companies and their movements between companies and so on.

My guess as described in the post Who is working where doing what? is that this will increasingly include social media data.

Product Master Data

Some of the same transactions or sums hereof in the data warehouse will have keys referencing products. These products will exist in the master data hub as members of various hierarchies with different categorizations.

My guess is that future developments in this field will further embrace not just your own products but also competitor products and market data available in the cloud all attached to your hierarchies and categorizations.   

Bookmark and Share

Fuzzy Hierarchy Management

Henrik Gabs Liliendahl6 Comments

When evaluating results from automated data matching your goal is typically to find false positives and false negatives being entities that are matched, but shouldn’t be (false positives) and entities that are not matched, but should have been (false negatives).

However the fuzziness often used in the data matching process also apply to the evaluation of the results as many dubious results isn’t a question about if the matched database rows are reflecting the same real world entity but more a question about if the matched (or not matched) database rows are reflecting different members of a real world hierarchy.

Example 1:

John Smith on 1 Main Street in Anytown
Mary & John Smith on 1 Main Str in Anytown

Example 2:

Anytown Municipality, Technical Dept
Municipality of Anytown

Example 3:

Acme Corporation, Anytown
Acme Corporation, Anywhere

All three examples above may be considered a false positive if matched and a false negative if not matched.

You may say that it depends on the purpose of use, which is true.

But if we are talking master data management we may probably encompass multiple requirements where we simultaneously need the match and don’t want the match, which is why we need to be able to resolve and store the results from fuzzy data matching into hierarchies.

Bookmark and Share

Single Business Partner View

Henrik Gabs Liliendahl4 Comments

If you search in google for “single customer view” you’ll get over 20,000 hits. If you search for “single business partner view” you’ll get zero – until I just posted this blog post.

Some time ago I wrote about getting a 360° Business Partner View elaborating on extending the 360° Customer View or Single Customer View (SVC) to embrace all sorts of party master data managed within the organization.

In fact there is at least the same amount of similar techniques used between

  • managing supplier master data and business-to business (B2B) customer master data

as there is between

  • managing business-to-business (B2B) customer master data and business-to-consumer (B2C) customer master data.

If you look at Customer Relation Management (CRM) systems almost every package is aimed at managing B2B data as the data model and the functionality supports real world B2B structures and how the sales force and other employees interacts with B2B customers and prospects.

Interacting with B2C customers and prospects is much more diverse and often supported by operational systems specialized for the industry in question like solutions for financial services, healthcare and so on.

A business partner is a party acting in the role as customer, prospect, supplier, reseller, distributor, agent and other forms of partnership. Sometimes the same party is acting in several roles at the same time thus potentially being both on the Sell–side and Buy-side of Master Data Quality management.

As sell side and buy side has intersections within party master data, in some industries we may also go deeper into identity resolution and find intersections between B2B entities and B2C entities. I’ve described these matters in the post So, how about SOHO homes. The business case is that some products in some industries are aimed at the households of business owners and the small businesses at the same time. This is for example true for industries as banking, insurance, telco, real estate and  law.

All in all achieving a single view of business partners is a task going beyond traditional customer data integration (CDI) and stretching into areas traditionally belonging to Product Information Management (PIM). This is a business case for multi-domain master data management.

Bookmark and Share

What’s a Six Pack?

Henrik Gabs LiliendahlLeave a comment

I have earlier written about my Right the First Time enrolment at the local fitness club and how I geekingly are using the dashboard on the workout equipment to follow my Fitness Data.  

But it is probably (or actually certainly) too early to talk about the term “six pack” related to these efforts.

So let’s talk about a “six pack” related to master data management.

We may for example have a look at “a six pack of Carlsberg lager”.

Sometimes you may ask how many different products you are handling in a master data hub. In answering that question we here may come up with a lot of different numbers all being a Perfect Wrong Answer.

The real world isn’t flat. When dealing with product master data we certainly need to see the world in hierarchies as:

  • Carlsberg lager as such is a product with some attributes and some relations to the customers liking this product or not.
  • The product may be brewed in the original country of origin (Denmark) or at lot of other facilities around the world, thus making it a different product per supplier with respect to some attributes.
  • As a customer you buy the product in a certain packaging like a six pack of cans in a given size with a given label.

The bottom level presented here is what in data management terms is identified as a Stock Keeping Unit (SKU).

Oh, and consuming the last “six pack” is probably (or actually certainly) not good for achieving the first mentioned “six pack”.

Bookmark and Share

Customer Relationship Mess (CRM)

Henrik Gabs LiliendahlLeave a comment

I have several times witnessed how a sales department for a lot of good reasons has forced the implementation of a CRM (Customer Relationship Management) software package disconnected from the ERP (Enterprise Resource Planning) system and other applications where customer master data have been handled until then.

The good reasons have been that the current applications didn’t fit the business processes in a dynamic sales department and perhaps that the current monolithic enterprise solution was too inflexible for the business needs in sales.

While this move may have been a great success in sales force automation the downside is often that the single customer view has been limited to a single customer view seen from the windows in the sales department offices.

In order to have a 360 degree view of customer you have to cover all the view points in the enterprise embracing all departments being in contact with the customer and thereby accessing and maintaining customer master data.

Those who feel the pain when a company doesn’t maintain such a view is the customer and those who enjoys when a company have that view is the customer.

Lately I had two experiences as a customer. A bad experience facing a lousy approximately 110 degree customer view from a phone company and a well executed 360 degree view from an insurance company. Both cases haven’t been around one of my favorite subjects being identity resolution. Both companies have my citizen ID.

It is just so that some companies cares more about single department business needs than true customer relationship management. IT’s a mess.

Bookmark and Share