As discussed in my last post a core part of many Data Quality tools is Data Matching. Data Matching is about linking entities in or between databases, where these entities are not already linked with unique keys.
Data Matching may be deployed in some different ways, where I have been involved in the following ones:
External Service Provider
Here your organization sends extracted data sets to an external service provider where the data are compared and also in many cases related to other reference sources all through matching technology. The provider sends back a “golden copy” ready for uploading in your databases.
Some service provider’s uses a Data Matching tool from the market and others has developed own solutions. Many solutions grown at the providers are country specific equipped with a lot of tips and tricks learned from handling data from that country over the years.
The big advantage here is that you gain from the experience – and the reference data collection – at these providers.
Internal Processing
You may implement a data quality tool from the market and use it for comparing your own data often from disparate internal sources in order to grow the “golden copy” at home.
Many MDM (Master Data Management) products have some matching capabilities build in.
Also many leading Business Intelligence tool providers supplement the offering with a (integrated) Data Quality tool with matching capabilities as an answer to the fact, that Business Intelligence on top of duplicated data doesn’t make sense.
Embedded Technology
Many data quality tool vendors provide plug-ins to popular ERP, CRM and SCM solutions so that data are matched with existing records at the point of entry. For the most popular such solutions as SAP and MS CRM there is multiple such plug-in’s from different Data Quality technology providers. Then again many implementation houses have a favorite combination – so in that way you select the matching tool by selecting an implementation house.
SOA Components
The embedded technology is of course not optimal where you operate with several databases and the commercial bundling may also not be the actual best solution for you.
Here Service Oriented Architecture thinking helps, so that matching services are available as SOA components at any point in your IT landscape based on centralized rule setting.
Cloud Computing
Cloud computing services offered from external service providers takes the best from these two worlds into one offering.
Here the SOA component resides at the external service provider – in best case combining an advanced matching tool, rich external reference data and the tips and tricks for your particular country and industry in question.
Liliendahl on Data Quality 
Henrik,
Excellent summary of the deployment options for data matching solutions.
Most of my career has been spent working on Internal Processing with some Embedded Technology and SOA Components experience occasionally mixed in.
I find the Cloud Computing option not only to be the most intriguing, but also the ultimate future of the industry. Maintaining internal “golden copies” only increases the “digital distance” between these abstract description and the real-world entities that they attempt to describe.
Master data especially will be moving into the cloud, and in the future its management will be performed by the real-world entities themselves, who will become their own data stewards.
When this shift finally happens (and it will take a long time for many reasons – mostly non-technology-related defense of the status quo), organizations will be able to focus on managing the data they truly own – transaction data – and stop trying to “herd cats” (wasting time and money) attempting to manage data they don’t own.
Best Regards,
Jim
Jim, thanks, cloud computing will indeed change the game of data matching and master data management in the years to come.
You may divide the data held by an enterprise into 3 pots:
• Global data that is not unique to operations in your enterprise but shared with other enterprises in the same industry (e.g. product reference data) and eventually the whole world (e.g. business partner data and location data). Here “shared data in the cloud” will make your “single version of the truth” easier and closer to the real world.
• Bilateral data concerning business partner transactions and related master data. If you for example buy a spare part then also “share the describing data” making your “single version of the truth” easier and more accurate.
• Private data that is unique to operations in your enterprise. This may be a “single version of the truth” that you find superior to what others have found, data supporting internal business rules that make your company more competitive and data referring to internal events.