About half a year ago I wrote a blog post called Who is Responsible for Data Quality aimed at issues with having your data coming from another corporation and going to another corporation.
My point was that many views on data governance, data ownership, the importance of upstream prevention and fitness for purpose of use in a business context is based on an assumption that the data in a given company is entered by that company, maintained by that company and consumed by that company. But this is in the business world today not true in many cases.
Actually a majority of the data quality issues I have been around since then has had exactly these ingredients:
- When data was born it was under an outside data governance jurisdiction
- The initial data owners, stewards and custodians were in another company
- Upstream wasn’t in the company were the current requirements are formulated
At the point of data transfer between the two jurisdictional areas the data is already digitalized and often it is high volume of data supposed to be processed in a short time frame, so the willingness and practical possibilities for implementing manual intervention is very limited.
This means that one case of looking for technology centric solutions is when data is born outside your jurisdiction. Also you tend to deal with concrete data quality rather than fluffy information quality in this scenario. That’s a pity, as I like information quality very much – but OK, data quality technology is quite interesting too.
Liliendahl on Data Quality 
That’s a very good point Henrik.
And in todays world of web services and tightly coupled information supply chains its only going to be more and more true.
One technique that can help is to establish “profiling gateways” where as the information enters your jurisdiction you run an automated data profiling task to establish a baseline, and then another gateway as the data leaves. This allows you to understand what you are working with on the input, and ensures that you are improving things (or at least not causing new problems) for the consumers downstream from you.
No matter how its done, the key is to approach such integrated data flows as partnerships, and work to improve communication with all parties- and avoid the “blame game” where everyone tries to prove that data quality is someone else’s fault. Data quality is everyones responsibility, and if you’re not part of the solution, then you are part of the problem.
Great post!
When the need for a data governance program is discussed, we often only focus on the jurisdiction of the organization instituting data governance in order to manage its data as a corporate asset — without considering its dependencies on external data assets.
Perhaps we need not only a Data Governance Board, but also a Data Governance Union or a United Programs of Data Governance or a Union of Socialist Data Governance Republics.
Okay, maybe not that last one 🙂
Maybe we also need Data Passports, Data Border Patrols, Bad Data Lists (and a process for resolving false positives on the Bad Data Lists).
Thought-provoking post, Henrik!
Great point, beyond just the data governance implications of dealing with another company. Increasingly, we have to look at data from beyond regional or national borders. This is especially true in healthcare. Do you trust the data that comes from a competing hospital? Are you sure it’s the same patient? Jim is right in that you need solid data governance – in the guise of a DG board, policies, etc – to achieve this.
Nice work, as always!
Thanks a lot James, Jim and Crysta.
Indeed, bilateral agreements between nations are often based on trust – and border control.