In a recent post here on this blog it was discussed: What is Identity Resolution?
One angle was the interchangeable use of the terms “Identity Resolution” and “Entity Resolution”. These terms can be seen as truly interchangeable, as that “Identity Resolution” is more advanced than “Entity Resolution” or as (my suggestion) that “Identity Resolution” is merely related to party master data, but “Entity Resolution” can be about all master data domains as parties, locations and products.
Another term sometimes used in this realm is “Non-Obvious Relationship Awareness”. Also this term is merely related to finding relationships between parties, for example individuals at a casino that seems to do better than the croupiers. Here’s a link to a (rather old) O’Reilly Radar post on Non-Obvious Relationship Awareness.
So “Non-Obvious Entity Relationship Awareness” could be about finding these hidden relationships in a multi-domain master data scope.
An example could be non-obvious relationships in a customer/product matrix.
The data supporting this discovery will actually not be found in the master data itself, but in transaction data probably being in an Enterprise Data Warehouse (EDW). But a multi-domain master data management platform will be needed to support the complex hierarchies and categorizations needed to make the discovery.
One technical aspect of discovering such non-obvious relationships is how chains of keys are stored in the multi-domain master data hub.
Customer Master Data
The transactions or sums hereof in the data warehouse will have keys referencing customer accounts. These accounts can be stored in staging areas in the master data hub with references to a golden record for each individual or company in the real world. Depending on the identity resolution available the golden records will have golden relations to each other as they are forming hierarchies of households, company family trees, contacts within companies and their movements between companies and so on.
My guess as described in the post Who is working where doing what? is that this will increasingly include social media data.
Product Master Data
Some of the same transactions or sums hereof in the data warehouse will have keys referencing products. These products will exist in the master data hub as members of various hierarchies with different categorizations.
My guess is that future developments in this field will further embrace not just your own products but also competitor products and market data available in the cloud all attached to your hierarchies and categorizations.
One of the major strengths of a data warehouse is the ability to quickly run reports across multidimensional hierarchies.
The speed with which this information can be queried can give the impression that the data warehouse holds more “powerful” data than the transactional databases from which the data was derived – rather like a well written piece of software can give the impression that a computer has intelligence!
Paradoxically, one of the major weaknesses of a data warehouse is the fact that it is hierarchal in structure. It is trying to represent data from relational databases, which have a a more powerful network structure, in an hierarchical form. The way that this is achieved is by replicating child entities in each dimension of the warehouse.
Because of this duplication there is a real danger of creating false relationships, that is, ones that do not exist in the original data.
The other weaknesses to which data warehouses are prone is false relationships arising through violation of Fifth Normal Form , as I described in an article that I wrote for DataQualityPro.
In the old days of the Gold Rush, far too often the glittering nuggets that inexperienced and naive prospectors risked their lives for was merely “Fools Gold”. So it is today with the unwary who go Data Mining, where the glittering nuggets, such as Golden Relationships that seem to offer fantastic new insights are, all too often, both false and worthless.
Thanks John. You are right. As William Shakespeare wrote in the play The Merchant of Venice:
All that glisters is not gold;
Often have you heard that told